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(1)  The  abUity  to  judge  time  to  collision  with  an  approaching  object  is  high  in  central  vision  but 
considerably  poorer  in  peripheral  vision.  (2)A  pilot’s  time  to  collision  with  a  simulated  approaching 
helicopter  depends  on  die  relative  contributions  of  ownship  speed  and  target  speed.  (3)  Binocular 
judgements  of  the  direction  of  motion  in  depth  are  similar  for  motion  in  the  vertical  and  horizontal 
meridians.  (4)  Spatial  frequency,  temporal  frequency,  orientation  and  contrast  ate  processed 
independendy  and  in  parallel.  (5)  Orientation  discrimination  for  cyclopean  form  and  texture-defined 
form  can  be  as  acute  as  for  luminance-defined  form.  (6)  A  letter  test  for  measuring  the  ability  to  see 
and  recognize  texture-defined  form  has  been  made  freely  available.  (7)  A  physiologically-plausible 
model  of  the  recognition  of  texture-defined  letters  has  b^n  developed.  (8)  The  ability  to  recognize 
texture-defined,  motion-defined  and  luminance-defined  letters  can  be  independendy  lost  in  patients 
with  multiple  sclerosis.  (9)  Magnetic  brain  recording  has  been  used  to  identify  and  locate  an  audio¬ 
visual  convergence  area  in  himan  brained  centres  for  color-defined,  texture-defined  and  luminance- 
defined  form.  (10)  Mathematical  methods  for  obtaining  the  response  of  auditory  hair  cells  to  complex 
amplitude  modulated  tones  and  the  harmonic  distortion  of  a  pure  tone  caused  by  the  hair  cell  transducer 
function  have  been  developed.  Results  fit  empirical  data  closely. 
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1.  RESEARCH  OBJECTIVES 

The  research  objectives  for  three  years  were  listed  in  the  grant  proposal 
as  follows. 

1.1  Long  Term  Aims 

1.1.1  We  will  further  explore  the  role  of  the  channeling  hypothesis  by:  (a) 
identifying  new  visual  channels;  (b)  elucidating  rules  for  cue  combination  in 
rich  visual  environments;  (c)  advancing  understanding  of  eye-limb 
coordination  in  skilled  visual  performance  and  the  role  of  inter-individual 
variations  of  visual  sensitivities  in  limiting  skilled  visual  performance;  (c) 
specifying  visual  parameters  likely  to  be  important  in  both  training  eye-limb 
coordination  in  flight  simulators  and  in  predicting  flying  performance  in  real 
aircraft. 

1.1.2  We  will  use  neuromagnetic  recording  techniques  to:  (d)  identify  the 
brain  sites  of  different  kinds  of  visual  processing,  auditory  processing  and 
multisensory  integration,  and  relate  these  sites  to  the  organization  of  visual, 
auditory  and  multisensory  convergence  areas  in  macaque  monkey  cortex;  (e) 
relate  objective  data  on  visual  and  auditory  processing  in  human  brain  to 
psychophysical  models  of  human  vision  and  hearing, 

1.2  Specific  Aims:  Psychophysics  in  Normally-Sighted  Individuals 

1.2.1  Camouflage  and  the  Visual  Processing  of  Objects  Defined  by  Binocular 
Disparity,  Relative  Motion  and  Luminance  Contrast 


1 


Department  of  Psychology 
FAX:  (416)  736-5814 


UNIVERSITE 


FACULTY  OF  ARTS 

4700  KEELE  STREET  .  NORTH  YORK  •  ONTARIO  .  CANADA  .  M3J  1P3 


We  will  determine  spatial  discrimination  thresholds  and  the  dynamics 
of  these  thresholds  for  spatial  form  defined  by  disparity  alone  over  a  wide 
range  of  crossed  and  uncrossed  disparities.  We  will  compare  these 
discrimination  thresholds  with  corresponding  discrimination  thresholds  for 
spatial  form  defined  by  luminance  contrast  alone,  for  spatial  form  defined  by 
motion  contrast  alone  and  for  spatial  form  defined  by  texture  contrast  alone 
over  the  same  range  of  disparities. 

Drawing  closer  to  the  everyday  visual  world,  we  will  measure  spatial 
discrimination  thresholds  for  form  that  is  defined  by  two  rather  than  one 
attribute  (specifically  disparity  plus  relative  motion  and  disparity  plus 
luminance  contrast). 

Finally,  by  means  of  cross-adaptation  and  cross-masking  paradigms,  we 
will  investigate  whether  the  detection  and  discrimination  of  spatial  form 
defined  by  disparity  alone,  luminance  contrast  alone  and  relative  motion 
alone  are  mediated  by  different  neural  mechanisms. 

1.2.2  Camouflage  and  the  Visual  Processing  of  Objects  Defined  by  Texture 
Contrast 

Using  our  texture-defined  letter  stimulus  we  will  compare  intersubject 
differences  in  thresholds  for  detecting  and  for  reading  letters  defined  by 
texture  contrast  for  a  population  of  20  normally-sighted  individuals.  We  will 
carry  out  measurements  for  static  texture  and  also  for  dynamic  texture  where 
a  new  texture  pattern  is  created  up  to  70  times  per  sec. 
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1.2.3  Cyclopean  Processing  of  Motion-in-Depth,  and  Changing-Size 

We  will  generate  targets  that  are  visible  in  binocular  view  but  invisible 
monocularly,  and  compare  the  sensitivities  and  dynamic  characteristics  of  the 
processing  of  motion-in-depth  generated  by  changing-disparity  and  changing- 
size  for  such  Cyclopean  targets.  By  comparing  these  data  with  corresponding 
data  for  noncyclopean  targets  we  will  distinguish  between  the  processing  of 
motion-in-depth  and  changing-size  that  occurs  before  and  after  convergence 
of  the  signals  from  left  and  right  eyes. 

1.2.4  Visual  Judgement  of  Time  to  Collision 

First,  we  will  extend  our  work  on  time-to-collision  thresholds  on 
foveally-viewed  stimuli  to  map  peripheral  visual  fields  for  time-to-collision 
thresholds. 

Second,  by  using  cyclopean  stimuli  we  will  dissociate  the  processing  of 
time  to  collision  information  that  occurs  before  and  after  binocular 
convergence. 

Third,  we  will  find  how  time-to-collision  judgements  are  affected  by 
imbalance  between  the  rate  of  expansion  of  retinal  image  size  and  retinal 
image  texture. 

1.2.5  Monocular  and  Binocular  Cues  to  the  Direction  of  Motion  in  Three 
Dimensions 

We  will  compare  a  subject's  ability  to  judge  the  direction  of  motion  in 
depth  when  (a)  binocular  and  monocular  cues  are  available  with  (b)  only  the 
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binocular  cue  is  available  and  (c)  only  the  monocular  cue  is  available.  We  will 
also  elucidate  how  the  visual  system  combines  these  two  cues.  (The  binocular 
cue  is  the  ratio  of  retinal  image  velocities  in  left  and  right  eyes,  and  the 
monocular  cue  is  (d<j)/d0)  as  discussed  on  page  44  -46).  We  have  previously 
described  the  processing  of  motion-in-depth  using  the  binocular  cue  alone 
(Refs.  34, 49)  and  the  monocular  cue  alone  (Refs.  154,  220,  230,233). 

1.3  Specific  Aims:  Psychophysics  in  Subjects  With  Known  Visual 
Disorders 

1.3.1  Detection  and  Recognition  of  Letters  Defined  by  Texture,  Motion  and 
Luminance 

Here  we  take  a  different  approach  to  the  same  question  as  that 
addressed  in  1.2.1  (paragraph  3)  above,  i.e.  whether  the  detection  and 
discrimination  of  three  kinds  of  spatial  form  involve  different  neural 
mechanisms.  Rather  than  testing  normally-sighted  individuals,  we  will 
compare  the  ability  of  selected  neurological  patients  to  detect  and  recognise 
texture-defined  letters,  motion-defined  letters  and  luminance-defined  letters. 

1.3.2  Binocular  Processing  of  Texture-Defined  Form,  Motion-Defined  Form 
and  Luminance-Defined  Form 

We  will  find  whether  binocular  processing  differs  for  three  kinds  of 
form  by  measuring  monocular  and  binocular  acuity  for  texture-defined 
letters,  motion-defined  letters  and  luminance-defined  letters  in  normally- 
sighted  subjects  and  in  patients  with  unilateral  focal  cerebral  lesions. 
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1.4  Specific  Aims:  Theoretical 

1.4.1  Multi- Rectifier  Models  of  the  Auditory  Pathway 

We  will  extend  our  theoretical  work  on  the  response  of  the  auditory 
hair  cell  to  amplitude-modulated  tones  by  developing  a  theoretical  treatment 
of  multi-rectifier  models  for  inputs  consisting  of  sums  of  amplitude- 
modulated  tones. 

1.4.2  Multi-Rectifier  Models  of  the  Visual  Pathway 

We  will  extend  our  theoretical  work  on  multi-rectifier  models  of  the 
visual  pathway  (Refs.  179, 183)  to  include  dynamics. 

1.4.3  Multi-Channel  Model  of  the  Detection  and  Discrimination  of  Texture- 
Defined  Form 

We  will  develop  parallel-process  models  of  the  detection  and  spatial 
discrimination  of  texture-defined  form  and  compare  these  models  against 
psychophysical  data  that  we  will  collect  on  a  subject's  ability  to  detect  and 
discriminate  texture-defined  letters. 

1.5  Specific  Aims:  Experimental  Neuromagnetic  Studies 
1.5.1  Visual  and  Auditory  Projections 

We  will  identify  evoked  activity  in  different  visual,  and  auditory 
projections  in  human  cortex,  elucidate  the  differences  between  the  type  of 
processing  occiurring  in  the  different  areas,  and  link  these  data  with  the 
known  functional  neuroanatomy  of  macaque  monkey  brain  and  with  human 
psychophysics. 
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1.5.2  Multi-Sensory  Areas 

We  will  identify  the  cortical  sites  of  interactions  between  responses  to 
stimuli  of  different  modalities,  and  compare  these  sites  with  the  known 
multi-sensory  cortical  areas  in  nonhuman  primates  (Van  Essen  et  al.,1990; 
Zeki,  1990). 

1.6  Specific  Aims:  Development  of  Operational  Tests 

1.6.1  Glare  Susceptibility 

We  will  find  whether  our  laboratory  procedure  for  quantifying 
intersubject  differences  of  susceptibility  to  glare  predicts  intersubject 
differences  in  flying  safety  in  glare  conditions. 

1.6.2  Collision  Avoidance 

We  will  find  whether  our  laboratory  procedure  for  quantifying 
intersubject  differences  in  the  ability  to  judge  time  to  collision  predicts 
intersubject  differences  in  flying  safety  in  realistic  visual  environments. 

2.  STATEMENT  OF  WORK 

The  statement  of  work  for  three  years  was  set  out  in  the  grant  proposal 
as  follows. 

2.1  Experiment  1.  Psychophysics:  Aspect  ratio  discrimination  for  cyclopean, 
luminance-defined  and  motion-defined  rectangles. 

Using  temporal  two-alternative  forced  choice  we  will  measure  aspect 
ratio  (shape)  discrimination  for  a  rectangular  area  within  a  region  of  random 
dots.  The  rectangular  area  will  be  perfectly  camouflaged  in  monocular  vision. 
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and  rendered  visible  by  binocular  disparity  alone.  Aspect  ratio  discrimination 
will  be  defined  as  the  just  noticeable  (75%  correct)  difference  in  the  ratio  a/b, 
where  a  and  b  are,  respectively,  the  rectangle's  horizontal  and  vertical  side 
lengths.  To  ensure  that  aspect  ratio  judgements  are  based  on  a  comparison  of 
side  lengths  rather  than  the  value  of  a  or  &  alone,  the  area  of  the  stimulus 
rectangle  will  be  varied  randomly  between  successive  presentations. 

Measurements  will  be  carried  out  over  a  wide  range  of  crossed  and 
uncrossed  disparities  for  static  and  for  dynamic  random  noise.  Ocular 
convergence  will  be  monitored  using  nonious  lines. 

A  luminance-defined  rectangle  with  the  same  spatial  sampling  as  the 
stereo  rectangle  will  be  created  by  switching  off  all  dots  outside  the  rectangle, 
and  the  above  measurements  repeated. 

A  motion-defined  rectangle  will  be  created  by  moving  the  dots  inside 
and  outside  the  rectangle  at  equal  and  opposite  speeds,  and  the  above 
measurements  repeated. 

2.2  Experiment  2.  Psychophysics:  Orientation  discrimination  for  cyclopean, 
luminance-defined  and  motion-defined  bars. 

The  procedures  described  above  will  then  be  repeated  except  that  orientation 
discrimination  for  a  bar  will  be  measured. 

2.3  Experiment  3:  Psychophysics:  Vernier  acuity  for  cyclopean,  luminance- 
defined  and  motion-defined  spatial  form. 
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The  procedures  described  in  2.2.1  will  be  repeated  except  that  vernier 
acuity  for  a  bar  will  be  measured. 

2.4  Experiment  4,  Psychophysics:  Aspect  ratio  discrimination,  orientation 
discrimination  and  vernier  acuity  for  spatial  form  defined  by  a  combination 
of  disparity  and  luminance  contrast. 

The  procedures  described  in  2.2.1  to  2.2.3  will  be  repeated  with  the 
stimuli  defined  by  a  combination  of  disparity  and  luminance  contrast. 

2.5  Experiment  5.  Psychophysics;  Cross-adaptation  of  aspect  ratio  aftereffect 
between  cyclopean,  luminance-defined  and  motion-defined  form. 

We  previously  discovered  an  aspect  ratio  adaptation  aftereffect  for 
luminance-defined  form  that  transferred  from  rectangles  to  ellipses 
independently  of  size  (Ref.  211).  Here  we  will  adapt  to  a  luminance-defined 
rectangle  and  test  for  an  aspect  ratio  aftereffect  with  luminance-defined, 
disparity-defined  and  motion-defined  rectangles  by  constructing  psychometric 
functions  as  in  Ref.  211.  Then  we  will  repeat  the  procedure,  but  adapting  to  a 
disparity-defined  rectangle  and  then  again,  but  adapting  to  a  motion-defined 
rectangle. 

2.6  Experiment  6.  Psychophysics:  Recognition  and  detection  of  texture- 
defined  letters. 

Using  temporal  2AFC,  two  texture  patterns  will  be  presented,  one 
containing  a  texture-defined  letter  and  the  other  no  letter  (See  section  6.1.1). 
The  subject's  tasks  will  be  (a)  identify  the  letter  and  (b)  state  whether  the  letter 
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was  presented  in  the  first  or  second  interval.  Ten  letters  will  be  presented. 

This  procedure  will  be  repeated  with  one  noise  dot  per  texture  line  degrading 
visibility,  then  with  two  noise  dots  per  letter  line,  and  so  on.  Thresholds  will 
be  estimated  from  psychometric  functions.  Measurements  will  be  made  for  1 
sec  presentations  with  the  same  texture  pattern  throughout  the  1-sec 
presentation,  with  ten  new  patterns  per  sec  and  with  70  new  patterns  per  sec. 
Ten  subjects  will  be  used. 

2.7  Experiment  7.  Psychophysics:  Monocular  and  binocular  cues  to  the 
perception  of  motion  in  depth;  comparison  of  results  for  cyclopean  and 
monocularly-visible  targets. 

We  will  generate  two  dot  patterns  and  view  them  dichoptically.  A  new 
dot  pattern  will  be  generated  at  50  times  per  sec  so  as  to  create  dynamic 
random  noise  and  one  pattern  will  be  routed  optically  to  the  left  eye  and  the 
other  pattern  to  the  right  eye.  It  is  necessary  to  use  dynamic  rather  than  static 
random  noise  because  a  moving  target  in  static  random  noise  breaks 
camouflage  and  would,  therefore,  be  visible  monocularly,  while  a  moving 
target  in  dynamic  random  noise  is  not  visible  monocularly  (Julesz,  1971). 

A  rectangular  area  within  the  dot  pattern  will  be  rendered  visible  by  a 
relative  displacement  of  the  left  and  right  eye's  rectangles  so  as  to  create 
binocular  disparity  as  described  by  Julesz  (1971).  First  we  will  modulate  the 
rectangle's  size  sinusoidally  and  using  temporal  2AFG,  separately  measure 
the  oscillation  amplitude  that  gives  just-detectable  size  oscillation  and  just 
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detectable  motion-in-depth  for  a  range  of  oscillation  frequencies  from  0.2  Hz 
upwards.  Then  we  will  oscillate  disparity  at  fixed  size  and  use  2AFC  to 
measure  the  amplitude  of  disparity  oscillation  for  just-detectable  motion-in¬ 
depth.  These  experiments  will  be  repeated  at  fixed  values  of  mean  disparity 
(crossed  and  uncrossed). 

A  pilot  experiment  using  our  new  hardware  dot  generator  indicates 
that  motion-in-depth  sensation  generated  by  changing-disparity  can  be 
canceled  by  pitting  changing-disparity  against  changing-size  stimulation.  This 
pilot  result  suggests  that  changing-size  and  changing-disparity  signals 
converge  at  a  single  motion-in-depth  stage.  For  sinusoidally  oscillating 
disparity  and  changing-size  we  will  use  temporal  2AFC  to  measure  the 
amplitude  of  size  oscillation  that  cancels  the  sensation  of  motion-in-depth 
produced  by  a  fixed  amplitude  of  disparity  oscillation  for  different  oscillation 
frequencies.  This  experiment  will  be  repeated  for  different  fixed  amplitudes 
of  disparity  oscillation.  Then  we  will  investigate  dynamics  in  the  time 
domain  using  transient  (ramping)  changes  of  disparity  and  changing-size. 

We  will  use  temporal  2AFC  to  measure  formally  the  ramping  rates  of  change 
of  size  that  cancels  the  sensation  of  motion-in-depth  a  fixed  rate  of  change  of 
disparity  for  different  ramp  durations.  This  experiment  will  be  repeated  for 
different  fixed  rates  of  change  of  disparity. 

The  above  experimental  protocols  are  the  same  as  these  used  earlier  for 
monocularly-visible  targets  (Ref.  85).  By  comparing  our  cyclopean  data  with 
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data  for  monocularly-visible  dotted  targets  we  will  endeavour  to  distinguish 
between  the  processing  underlying  motion-in-depth  perception  that  takes 
place  after  and  before  convergence  of  the  signals  from  left  and  right  eyes. 

2.8  Experiment  8.  Psychophysics:  Visual  fields  for  time  to  contact 
judgements. 

This  experiment  follows  on  from  Ref.  220.  Consider  a  rigid  object 
moving  directly  towards  the  eye  at  constant  speed.  The  relation  between  the 
angular  size  of  the  object's  retinal  image  (0t)  and  time  (t)  is  given  by 


tan  00 
tan  0t  =  i.t/To 


(1) 


We  will  use  our  special  purpose  hardware  to  create  a  retinal  image  that 
exactly  mimics  an  approaching  object.  Ten  different  values  of  time  to 
collision  will  be  where  To  is  time  to  collision  at  time  t  =  0.  As  described 


previously  (Ref.  220).  presented,  and  subjects  will  be  instructed  to  press  one 
button  if  the  simulated  object  would  arrive  sooner  than  the  mean  of  the 
stimulus  set  and  a  second  button  if  it  would  arrive  later  than  the  mean  of  the 


set  (a  procedure  first  described  by  McKee,  1981).  As  described  below  (Section 
7.5,  page  69  -  78)  we  will  generate  a  8  x  8  stimulus  matrix  and  analyse  it  in  two 
ways  so  as  to  demonstrate  that  subjects  base  their  judgements  on  variations  in 
0/(d0/dt)  rather  than  variations  in  (d0/dt). 

We  construct  a  visual  field  perimeter  with  nonious  line  control  of 
vergence  and  use  it  to  measure  discrimination  threshold  for  time  to  contact  at 
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the  fovea  and  the  following  retinal  locations:  2°,  4°,  8°,  16°  left  and  right,  up 
and  down  and  the  two  obliques.  Similar  measurements  will  be  made  for  5 
subjects  for  mean  times  to  collision  of  1  sec,  2  sec  and  4  sec. 

2.9  Experiment  9.  Psychophysics:  Judgement  of  time  to  contact  for  textured 
targets;  effect  of  mismatching  rate  of  expansion  of  texture  and  object  size. 

As  in  2.8  we  will  generate  a  retinal  image  whose  rate  of  change  of  size 
exactly  mimics  the  retinal  image  of  a  rigid  approaching  object.  This  time, 
however,  the  retinal  image  will  be  textured.  We  will  design  an  instrument 
based  on  an  earlier  design  (Ref.  125)  that  will  create  texture  whose  density  can 
be  varied  by  indefinitely  small  amounts.  We  will  measure  discrimination 
threshold  for  time  to  collision  with  rate  of  expansion  of  texture  and  rate  of 
expansion  of  size  matched  exactly  as  in  real-world  conditions,  and  then  with  a 
range  of  mismatch  conditions.  Specifically,  we  will  use  the  following  values 
of  the  ratio  (rate  of  expansion  of  texture) /(rate  of  expansion  of  size)  in  the 
retinal  image  of  the  simulated  object:  0.8;  0.9;  1.0;  1.2;  1.4. 

2.10  Experiment  10.  Psychophysics:  Comparison  of  time  to  collision 
judgements  for  cyclopean  and  monocularly-visible  targets. 

As  in  2.8  we  will  generate  a  retinal  image  whose  rate  of  change  of  size 
exactly  mimics  the  retinal  image  of  a  rigid  approaching  object.  However,  in 

2.10  we  will  use  our  stereo  dynamic  random  noise  stimulator  to  ensure  that 
the  simulated  object  is  perfectly  camouflaged  monocularly,  and  only  visible 
in  binocular  view.  This  will  ensure  that  all  processing  of  time  to  collision 
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takes  place  in  cyclopean  vision,  i.e,  after  the  signals  from  the  left  and  right 
eyes  have  converged.  Discrimination  threshold  for  time  to  contact  will  be 
measured  using  our  8x8  matrix  method  as  in  6.2.8.  Then  we  will  switch  off 
all  dots  outside  the  camouflaged  stimulus  so  as  to  allow  it  to  be  seen 
monocularly,  and  repeat  the  experiment.  By  comparing  the  cyclopean  and 
non-cyclopean  thresholds  we  will  attempt  to  separate  the  processing  of  time 
to  collision  that  occurs  before  and  after  convergence  of  the  signals  from  left 
and  right  eyes. 

2.11  Experiment  11.  Psychophysics:  Discrimination  of  the  direction  of 
motion  in  depth  in  different  meridia  using  monocular  and  binocular  cues. 

We  will  generate  two  bright  squares  using  our  special-purpose 
hardware  (Ref.  220),  and  route  one  to  the  left  eye  and  one  to  the  right  eye. 
Stereo  motion  in  depth  will  be  created  by  translating  the  squares  on  the 
Tektronix  608  monitors,  and  the  direction  of  motion  in  depth  will  be 
simulated  straightforwardly  by  adjusting  the  relative  velocity  of  these 
translations  as  described  previously  (Refs.  34,  49).  Simulating  the  monocular 
direction  of  motion  in  depth  is  more  difficult,  as  we  discuss  next. 

We  have  shown  that  an  object  moving  along  a  straight  line  in  depth 
creates  a  retinal  image  whose  angular  location  {^\)  on  the  retina  varies  with 
time  (t)  as  follows 

n  tan  00 

ITo  /  t)-1  . 
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where  n  is  the  number  of  object  linear  radii  by  which  the  centre  of  the  object 
misses  the  eye's  first  nodal  point,  and  0^  is  the  object's  angular  radius  at  time 

t=0.  At  the  same  time  the  size  of  the  retinal  image  grows  progressively  larger, 
as  given  by  equation  (1)  above,  i.e. 

tan  00 

tan0t=iT(tTT^ 

where  0t  is  the  objects'  angular  radius  at  the  eye  at  time  t=0  (Ref.  230). 

We  have  also  shown  that  the  direction  of  motion  in  depth  is  given  by 


By  means  of  arbitrary  waveform  generators  (Wavetek  Model  75)  we 
will  control  the  size  and  locations  of  the  squares  on  the  monitors  so  as  to 
exactly  mimic  the  retinal  images  of  an  approaching  object. 

Motion-in-depth  along  different  trajectories  will  be  mimicked  by 
moving  the  rectangle  across  the  CRT  screen  at  different  speeds  and  at  the 
same  time  changing  the  rectangle's  size.  The  resulting  stimulus  will  be 
quantified  in  terms  of  the  relative  velocity  of  the  rectangle's  opposite  edges. 
The  rectangle's  motion  will  be  controlled  by  a  PC  (2AFC,  method  of  constant 
stimuli).  We  will  measure  the  just  discriminable  change  of  trajectory  for  a 
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wide  range  of  trajectories  in  the  horizontal,  vertical  and  the  two  45  deg 
oblique  planes  for  a  population  of  20  control  subjects  in  the  following 
conditions,  (a)  binocular  and  monocular  cues  both  available  and  matched 
exactly  as  in  the  real  world  for  the  particular  viewing  distance  and  target  size; 
(b)  only  the  binocular  cue  available;  (c)  only  the  monocular  cue  available. 

2.12  Experiment  12.  Psychophysics:  Detection  and  recognition  of  texture- 
defined  letters,  motion-defined  letters  and  contrast-defined  letters  in  control 
subjects  and  in  patients  with  multiple  sclerosis. 

We  will  obtain  norms  for  detection  and  recognition  of  texture-defined 
letters,  motion-defined  letters  and  luminance-defined  letters.  We  will  use 
two-response  temporal  2AFC  along  the  lines  of  our  previous  work  with 
motion-defined  letters.  Then  we  will  test  20  patients  with  multiple  sclerosis. 
All  patients  will  have  acuity  of  6/12  or  better  and  normal  visual  fields. 

2.13  Experiment  13.  Psychophysics:  Detection  and  recognition  of  texture- 
defined  letters,  motion-defined  letters  and  luminance-defined  letters  in 
patients  with  neurosurgical  removal  of  brain  tissue. 

This  study  will  be  similar  to  2.13  except  that  the  patient  group  will  be  20 
patients  with  neurosurgical  removal  of  cerebral  tissue  imilaterally.  All 
patients  will  have  acuity  of  6/12  or  better  and  normal  visual  fields. 

2.14  Experiment  14.  Psychophysics:  Comparison  of  monocular  and 
binocular  thresholds  for  recognising  texture-defined  letters  in  patients  with 
documented  brain  lesions. 


YORK 

UNIVERSITY 


15 


Department  of  Psychology 
FAX:  (416)  736-5814 


^  UNIVERSITE 

A  YORK 


UNIVERSITY 
FACULTY  OF  ARTS 

4700  KEELE  STREET  .  NORTH  YORK  •  ONTARIO  .  CANADA  •  M3J  1P3 


A  group  of  20  control  subjects  will  carry  out  the  following  protocol:  (1) 
Measurement  of  noise  threshold  for  detecting  and  recognising  texture- 
defined  letters  in  binocular  vision;  (2)  measurement  of  speed  threshold  for 
detecting  and  recognising  motion-defined  letters;  (3)  measurement  of  contrast 
threshold  for  detecting  and  recognising  luminance-defined  letters.  All  the 
measurements  will  then  be  repeated  for  left  monocular  vision  and  right 
monocular  vision.  A  group  of  20  patients  with  documented  (CAT  scan  or 
MRI)  unilateral  focal  cerebral  lesions  will  then  carry  out  the  same  protocol. 
The  value  of  (binocular  threshold)/ (best  monocular  threshold)  will  then  be 
calculated  for  each  threshold  separately  and  each  separate  mean  and  SD 
calculated.  Then  we  will  find  whether  the  ratios  differ  significantly  for  any  of 
the  thresholds.  All  these  calculations  will  then  be  repeated  for  the  patient 
group. 

2.15  Experiment  15.  Neuromagnetism:  Comparison  of  brain  responses  to 
color-defined  and  luminance-defined  gratings. 

We  will  compare  electric  (VEP)  and  magnetic  (MEG)  responses  to 
chromatic  contrast  and  to  luminance  contrast.  Equiluminant  red  and  green 
gratings  will  be  generated  and  superimposed  in  either  180  deg  or  0  deg  spatial 
phase,  thus  giving  a  red-green  chromatic  grating  or  a  yellow  luminance 
grating.  The  dominant  wavelengths  of  the  red  and  green  lights  will  be 
selected  to  fall  along  one  of  the  cardinal  directions  of  color  space.  Grating 
contrast  will  be  abruptly  inaeased  and  decreased  with  one  sec  periodicity. 
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First,  response  amplitude  will  be  plotted  versus  red/green  ratio  on  either  side 
of  equiluminance  to  check  that  the  response  is  a  true  chromatic  response. 
Responses  produced  by  a  4  x  4  deg  chromatic  grating  presented  to  the  lower 
left  visual  macular  quadrant  will  be  recorded  from  about  56  coil  locations  and 
maps  constructed.  If  any  component  is  a  close  approximation  to  a  single 
dipole  distribution,  the  location  of  the  single  equivalent  dipole  will  be 
calculated.  This  procedure  will  then  be  repeated  for  the  remaining  three 
quadrants.  Then  the  entire  procedure  will  be  repeated  for  the  luminance 
grating.  Waveforms  and  source  locations  will  be  compared  and  discussed  in 
terms  of  the  visual  areas  in  macaque.  Then  the  entire  procedure  will  be 
repeated  for  a  yellow-blue  grating.  Three  color-normal  subjects  will  be  tested. 

Preliminary  experiments  show  that,  at  some  locations  on  the  head,  the 
MEG  response  to  red-green  chromatic  grating  is  larger  than  the  MEG  response 
to  the  luminance  grating  and  has  a  different  waveform. 

2.16  Experiment  16.  Neuromagnetism:  Objective  measurement  of 
orientation  and  spatial  frequency  tuning  of  human  brain  cells  sensitive  to 
color-defined  and  luminance-defined  gratings. 

We  will  generate  a  red-green  chromatic  grating  of  spatial  frequency 
Sic/deg  that  is  counterphase-modulated  at  Fi  Hz,  and  superimpose  this  on  a 

second  red-green  chromatic  grating  of  the  same  orientation  and  spatial 
frequency  that  is  counterphase-modulated  at  F2  Hz.  The  stimulus  field  will  be 

4  deg  in  diameter  and  placed  in  one  of  the  macular  quadrants.  The  intent  is  to 
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Stimulate  a  relatively  homogeneous  retinal  area  and  to  minimize 
complications  due  to  cortical  folding.  The  red  and  green  wavelengths  will  be 
chosen  to  fall  along  one  of  the  cardinal  axes  in  color  space.  While  subjects 
view  the  grating  we  will  subject  the  MEG  to  high-resolution  zoom-FFT.  By 
varying  the  relative  spatial  phase  of  the  two  gratings  we  will  search  for  cross¬ 
modulation  response  components  that  are  independent  of  relative  spatial 
phase,  and  thus  depend  on  the  spatial  frequency  spectrum  of  the  stimulus, 
but  not  on  any  particular  luminance  distribution  on  the  retina.  (We 
developed  this  approach  in  a  previous  study  on  luminance  gratings  and 
found  that  2Fi  +  2F2  was  such  a  component,  at  least  for  luminance  gratings). 

Such  a  component  must  be  generated  by  neurons  that  "see"  both  chromatic 
gratings.  To  study  the  spatial  frequency  bandwidth  of  neurons  sensitive  to  a 
vertical  Si  c/ deg  red-green  grating,  we  will  plot  the  amplitude  of  the 

designated  response  component  versus  the  spatial  frequency  of  the  second 
grating.  To  study  the  orientation  tuning  bandwidth  of  neurons  sensitive  to  a 
vertical  Si  c/ deg  grating,  the  orientation  of  the  second  grating  will  be  varied 
while  holding  its  spatial  frequency  a  little  different  from  Si  c/deg.  To  study 
the  temporal  tuning  bandwidth  of  neurons  sensitive  to  a  vertical  Si  c/deg 
grating  the  temporal  frequency  of  the  second  grating  will  be  varied  while 
holding  its  orientation  vertical  and  its  spatial  frequency  a  little  different  from 
Si  c/ deg.  We  will  then  measure  the  spatial  frequency,  orientation  and 

temporal  frequency  tuning  bandwidths  for  a  vertical  yellow  luminance 
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grating  of  frequency  Si  c/deg  using  the  above  procedure.  We  will  compare 
these  bandwidths  at  1.5  c/deg,  4  c/deg  and  8  c/ deg.  The  basis  of  this  approach 
is  described  in  Refs.  167  &  178  and  on  pp.  420  -  427  of  my  1989  book.  Results 
will  be  discussed  in  terms  of  the  corresponding  bandwidths  of  neurons  in 
monkey  visual  cortices. 

2.17  Experiment  17.  Neuromagnetism:  Objective  measurement  of  the 
responses  of  human  brain  cells  to  binocular  depth  and  stereomotion. 

For  MEG  experiments  it  is  difficult  to  use  a  cyclopean  depth  stimulus 
generated  by  conventional  passive  optics  as  in  6.2.1 — 6.2.5  above,  because  the 
MEG  dewar  requires  a  prone  position  of  the  subject  and  variable  head 
orientations.  It  is  not  possible  to  use  active  goggles  to  route  the  signals  to  left 
and  right  eyes  because  of  interference.  We  will  solve  the  problem  of  cyclopean 
stimulation  as  follows.  The  subject,  located  inside  the  magnetically-shielded 
room  will  wear  passive  circularly-polarized  goggles,  and  the  computer¬ 
generated  stimulus  will  be  viewed  through  a  liquid  crystal  cell  outside  the 
room  whose  sense  of  circular  polarization  will  be  switched  synchronously 
with  the  frame  rate.  By  using  dynamic  rather  than  static  random  dots  we  will 
ensure  that  transitions  between  depth  planes  are  not  visible  monocularly.  We 
will  generate  a  random  dot  pattern  containing  cyclopean  gratings  or  checks 
and  will  alternate  this  stimulus  with  a  zero-depth  dot  pattern.  We  will 
measure  average  transient  responses  for  MEG  and  EEG  for  crossed  and 
uncrossed  disparities  from  each  of  four  quadrants  of  the  visual  field.  A  range 
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of  check  sizes  and  spatial  frequencies  will  be  explored.  Ocular  vergence  will 
be  monitored  using  nonious  lines.  We  will  map  the  MEG  response  and,  if 
any  components  fit  a  single-dipole  field,  we  will  estimate  the  location  of  the 
equivalent  dipole. 

As  a  control,  we  will  carry  out  a  similar  experiment  for  vertical 
disparities.  Also,  we  will  induce  periodic  vergence  changes  and  map  the  time- 
averaged  MEG  response.  Three  stereo-normal  subjects  will  be  tested.  Results 
will  be  discussed  in  terms  of  single-unit  responses  to  stereo  stimulation  in 
macaque. 

In  further  experiments  we  will  find  which  areas  of  human  cortex  are 
selectively  sensitive  to  the  direction  of  stereomotion-in-depth  and  compare 
their  properties  with  the  properties  of  stereomotion  channels(Refs.  34,  49)  and 
motion-in-depth  neurons  (Refs.  67,  108,  109).  We  will  use  areas  of  motion-in- 
depth  "blindness"  in  control  subjects  to  check  that  neuromagnetic  responses 
to  changing  disparity  are  associated  with  motion-in-depth  perception. 

2.18  Experiment  18.  Neuromagnetism:  Objective  localization  of  an  audio¬ 
visual  convergence  area  in  human  brain  cortex. 

An  amplitude-modulated  (AM)  tone  of  modulation  frequency  Fi  Hz 
will  be  fed  to  a  subject's  ear  while  the  subject  views  a  light  flickering  at  F2  Hz. 

The  MEG  will  be  subjected  to  high-resolution  zoom-FFT  and  a  search  made 
for  terms  of  frequency  (nFi  ±  mF2).  These  terms  must  originate  in  neurons 

that  receive  both  auditory  and  visual  input.  We  will  then  map  these  terms.  If 
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any  map  approximates  a  single  dipole  field  we  will  compute  the  equivalent 
dipole  location,  thus  locating  an  audio-visual  convergence  area.  Results  will 
be  discussed  in  terms  of  known  audio-visual  convergence  areas  in  macaque. 

Pilot  experiments  have  already  revealed  an  (nFi  ±  MF2)  term,  but  it  was 
weak,  and  we  need  to  vary  Fi  and  also  F2  to  find  optimum  values. 

2.19  Experiment  19.  Operational  tests:  Evaluation  of  our  laboratory  test  for 
glare  sensitivity  in  predicting  intersubject  differences  in  pilot  performance  in 
high-glare,  low-contrast  conditions  using  ground-based  flight  simulators  and 
the  flying  simulator. 

We  will  administer  our  laboratory  glare  susceptibility  test  (Refs.  195  & 
208)  to  a  group  of  20  pilots.  These  pilots  will  then  use  a  flight  simulator  to  test 
their  ability  to  see,  while  landing,  objects  of  different  contrasts  without  glare 
and  in  two  kinds  of  high-glare  environments,  one  with  a  point  glare  source 
(simulating  the  sun)  and  one  with  diffuse  glare  (as  in  low  level  flight  over 
snow).  We  will  use  CAE  commercial  airline  heavy  jet  simulators,  the 
University  of  Toronto  simulator  and  the  NRC  of  Canada  flying  simulator. 

We  will  correlate  intersubject  differences  in  the  laboratory  and  tests  with 
intersubject  differences  in  the  flying  tests. 

2.20  Experiment  20.  Operational  tests:  Collision  avoidance  in  aviation;  pilot 
performance  studied  using  ground-based  flight  simulators  and  the  flying 
simulator. 
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We  will  administer  our  laboratory  tests  for  time  to  collision  and  for 
sensitivity  to  rate  of  expansion,  i.e.  looming  (Ref.  220)  and  our  motion- 
defined  letter  test  (Ref.  190)  to  a  group  of  20  pilots.  These  pilots  will  then  use  a 
flight  simulator  to  test  their  ability  to  avoid  air  to  air  collision  and  to  avoid 
collision  with  terrain  features  at  low  altitude.  We  will  use  the  CAE 
commercial  airline  heavy  jet  simulators,  the  University  of  Toronto  simulator 
and  the  NRC  of  Canada  flying  simulator.  We  will  compare  intersubject 
differences  in  the  laboratory  and  flying  tests. 

3  STATUS.  QE..mE..RBSEARCH..EFEQRT 

During  the  period  1993  to  December  1996, 37  papers  in  refereed 
Journals,  3  book  chapters,and  2  chapters  in  Conference  Proceedings  have  been 
published.  A  total  of  12  abstracts  were  presented  at  the  1994, 1995  and  1996 
ARVO  meetings.  Science  Citation  Index  counts  are  as  follows:  194  (1991),  251 
(1992),  239  (1993),  255  (1994),  329  (1995). 

Note:  This  report  is  organised  as  follows.  A  ground-based  simulator 
study  is  reported  in  Section  3.1  and  a  paper  on  disorientation  in  aviation  is 
summarized  in  Section  3.2.  Experimental  work  on  discriminating  the 
direction  and  speed  of  motion  in  depth  using  binocular  cues  only  is  reported 
in  Section  3.3.  Section  3.4  reports  a  comparison  of  foveal  and  peripheral 
discriminations  of  time  to  collision  with  an  approaching  object. 

In  a  series  of  previous  reports  I  described  work  on  the  visual  processing 
of  spatial  form  defined  by  luminance  contrast  and  the  visual  processing  of 
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spatial  form  defined  by  motion  contrast.  This  work  is  now  extended  to  the 
visual  processing  of  spatial  form  defined  by  binocular  disparity,  i.e.  cyclopean 
form  (Sections  3.7  -  3.8)  and  the  visual  processing  of  spatial  form  defined  by 
texture  contrast  (Sections  3.9  -  3.13).  Research  on  visual  channeling  and  the 
independence  of  visual  processing  is  reported  in  Sections  3.14  &  3.16. 

Magnetic  and  electrical  recording  from  the  human  brain  is  reported  in 
Sections  3.17  -  3.19.  Theoretical  and  experimental  research  on  the  auditory 
processing  of  complex  sounds  is  reported  in  Sections  3.20  -  3.21. 

3.1  Ground-based  flight  simulator  study. 

The  experiment  is  completed.  A  paper  entitled  "Collision  avoidance:  a 
helicopter  simulator  study"  by  R.  Kruk  &  D.  Regan  has  been  published  in 
Aviation,  Space  &  Environmental  Medicine  (1996)  67, 111-114  (Ref  254). 

The  broad  issue  here  is  collision  avoidance,  and  in  particular  "when 
will  it  hit  me?".  This  experiment  addresses  a  pilot's  ability  to  use  the  retinal 
image  correlate  of  time  to  collision  expressed  by  Hoyle's  equation 

T«e/(de/dt) -  (1) 

where  T  is  the  time  to  collision  with  a  rigid  nonrotating  object  of  angular 
diameter  6  approaching  at  constant  speed  along  the  line  of  sight  (Hoyle,  1957). 

It  will  be  useful  to  distinguish  the  following  four  kinds  of  collision 
between  a  helicopter  and  an  external  object  (e.g.  a  terrain  feature  or  a  second 
aircraft).  (1)  Ownship  stationary,  an  object  approaching  at  constant  speed  on  a 
collision  course.  (2)  Ownship  moving  at  constant  speed  towards  a  stationary 
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object.  (3)  Ownship  and  an  object  moving  towards  each  other,  both  at 
constant  speed.  (4)  Ownship  and  an  object  moving  on  the  same  course  in  the 
same  direction,  both  at  constant  speed  but  ownship  moving  faster  than  the 
object. 

The  experiment  was  carried  out  using  the  Simulator  Training  Research 
Advanced  Testbed  for  Aviation  (STRATA)  at  the  Army  Research  Institute 
(ARI)  at  Fort  Rucker,  Alabama.  The  STRATA  is  a  research  simulator  based 
upon  the  AH-64  Apache  Helicopter. 

The  display  system  used  was  a  CAE  fibre-optic-helmet-mounted-display 
system  (FOHMD)  providing  a  low  resolution  field  (5  arcmin/pixel)  66° 
vertical  by  82.5°  horizontal  to  each  eye,  overlapped  38°  to  provide  a  total 
instantaneous  field  of  view  (FOV)  of  66°  x  127°.  A  high  resolution  field  (1.5 
arcmin/pixel)  19°  vertical  by  25°  horizontal  is  centered  in  the  overlap  region. 
In  the  present  study  3  FOV  were  used;  FOVl  was  66°  x  127°  with  centered  19°  x 
25°  high  resolution  insert.  FOV2  was  19°  x  25°  high  resolution  insert  only. 
FOV3  was  3°  X  3°  in  the  centre  of  the  high  resolution  insert.  While  the 
system  is  capable  of  stereoscopic  image  presentation,  in  the  present  study  the 
image  was  bi-ocular  (i.e.  exactly  the  same  image  presented  to  each  eye  in  the 
overlap  area).  Average  scene  luminance  with  the  daytime  database  used  for 
the  study  was  30  Ft.L. 

The  image  was  a  real  time  (60  Hz  update)  of  a  valley  near  MESA, 
Arizona  in  daytime.  It  was  generated  by  an  Evans  and  Sutherland  ESIG  1000 
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Image  Generator.  The  targets  were  a  grey  CH-53  heavy  helicopter  and  a  green 
OH-58  light  helicopter. 

The  altitude  of  own  aircraft  and  the  target  was  always  15  ft  above 
ground.  The  pilot  initiated  each  trial  by  pressing  a  button  on  the  cyclic 
(control  column).  From  time  0  to  +10  seconds  ownship  either  remained  in  a 
hover,  accelerated  to  30  knots  (50  ft/ sec,  30.5  m/ sec)  or  accelerated  to  60  knots 
(100  ft/sec,  60.1  m/sec).  Ownship  speed  was  constant  from  time  +10  to  +15  sec 
at  0, 30  or  60  knots.  At  +15  seconds  the  target  appeared  at  a  range  of  300, 400, 
600, 800  or  1200  feet  (91.4, 122, 183, 244  or  366  m)  and  it  could  be  in  the  hover, 
moving  towards  ownship  at  30  knots  or  60  knots,  or  flying  backwards  at  30 
knots.  Closing  speed  could  be  30,  60,  or  90  knots.  Maximum  time  between 
target  appearance  and  collision  was  8  seconds,  and  minimum  was  5.5  seconds. 
At  either  2, 2.5  or  3  seconds  before  collision  (randomly  selected)  the  target 
disappeared.  Pilots  were  instructed  to  extrapolate  the  closure  rate  of  the  target 
and  press  the  trigger  on  the  cyclic  at  the  time  when  collision  would  have 
occurred.  Two  seconds  after  the  trigger  pull  was  recorded  the  system 
reinitialized  the  ownship  to  the  hover  and  a  2000  Hz  tone  would  soimd  to 
indicate  that  another  trial  could  commence  at  pilot  command.  Six  qualified 
U.  S.  Army  Helicopter  pilots,  with  experience  ranging  from  470  to  1650  hours 
(mean  765)  participated  in  the  study. 
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Figure  1  Estimated  time  to  collision  in  msec  (ordinate)  versus  field  of 
view  (FOV).  FOVl,  FOV2  and  FOV3  were,  respectively,  66  (vertical)  x  127°,  19 
X  25°  and  3  x  3°.  Filled  symbols:  ownship  speed  60  knots,  target  stationary, 
closing  speed  60  knots.  Open  symbols:  ownship  stationary,  target  speed  60 
knots,  closing  speed  60  knots.  (60  knots  is  61  m/sec). 
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The  difference  in  time  (milliseconds)  and  range  (feet)  between  pilot's 
trigger  press  and  (system  calculated)  actual  collision  were  collected  for  each 
trial.  These  data  were  subjected  to  a  repeated  measures,  within  subjects 
analysis  of  variance  (ANOVA). 

Fig.  1  compares  findings  in  the  conditions  that  closing  speed  was  produced 
either  entirely  by  ownship  motion  or  entirely  by  target  motion.  Data  were 
collapsed  for  both  targets  and  for  all  six  pilots.  Estimated  times  of  collision  are 
plotted  as  ordinate.  Zero  on  the  ordinate  is  the  time  at  which  collision  with 
the  target  aircraft  would  have  occurred.  The  three  conditions  plotted  along 
the  abscissa  are  the  large,  intermediate  and  small  fields  of  view  (FOVl,  FOV2 
and  FOV3  respectively).  All  pilots  reported  that,  in  the  large  and 
intermediate  field  of  view  conditions,  ownship  motion  resulted  in  a  strong 
sensation  of  self-motion.  In  the  smallest  field  of  view  condition,  however, 
even  the  highest  ownship  speed  produced  little  or  no  sensation  of  self- 
motion. 

Filled  symbols  in  Fig.  1  plots  estimated  collision  times  for  a  stationary 
target  and  ownship  forward  speed  of  60  knots  (30.5  m/sec).  In  the  large  and 
intermediate  field  of  view  conditions,  pilots  estimated  that  collision  with  the 
stationary  target  would  occur  roughly  200  msec  before  the  actual  time  of 
collision.  Findings  were  different  in  the  case  of  ownship  stationary  and  the 
target  approaching  at  60  knots  (open  symbols).  In  that  case,  estimate  of 
collision  time  showed  considerably  less  lead  in  the  large  and  intermediate 
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field  of  view  conditions.  When  the  smallest  field  of  view  was  used,  estimates 
were  the  same  whether  the  60  knot  closing  speed  was  produced  entirely  by 
ownship  motion  or  entirely  by  target  motion.  Both  lagged  by  roughly  150 
msec.  A  possible  reason  for  the  lag  is  that  the  small  field  of  view  was  so  small 
that  the  angular  size  of  the  larger  target  exceeded  the  field  of  view  during  part 
of  its  approach  so  that  the  retinal  image  information  expressed  in  equation  (1) 
would  have  been  degraded. 

Our  main  finding  is  that  a  pilot's  estimate  of  time  to  collision  depends 
on  the  relative  contributions  of  closing  speed  to  ownship  speed  and  of  target 
speed.  For  example,  estimates  of  time  to  collision  lead  more  when  a  60  knot 
(61  m/ sec)  closing  speed  is  produced  entirely  by  ownship  motion  than  when 
it  is  produced  by  target  motion  when  our  66°  x  127°  and  19°  x  25°  fields  of 
view  are  used.  This  is  surprising:  one  would  not  necessarily  expect  any 
difference  in  time  to  contact  estimation  in  the  ownship-only  motion  and 
target-only  motion  conditions  because  the  closing  speeds  are  identical  and  an 
accurate  retinal  image  correlate  of  time  to  contact  is  available  in  both 
conditions  (see  equation  1).  We  assume  that  too-short  estimates  are  caused  by 
the  vection-induced  impression  of  self-motion  that  is  present  in  the 
ownship-only  motion  but  not  in  the  target-only  motion  condition.  This 
suggestion  is  consistent  with  the  finding  that  estimates  in  the  two  conditions 
were  similar  when  the  smallest  field  of  view  was  used;  vection  was  weak  or 
absent  for  the  smallest  field  of  view. 
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3.2  A  review  entitled  "Spatial  orientation  in  aviation:  Visual 
contributions"  by  D.  Regan  has  been  published  in  the  Journal  nf  Vestihular 
Research  (Ref.  248). 

3.3  Binocular  judgements  of  the  direction  of  an  approaching  nhiprt 

One  paper  entitled  "Binocular  correlates  of  the  direction  of  motion  in 
depth"  by  D.  Regan  has  been  published  in  Vision  Research.  1993, 33, 2359- 
2360.  A  second  paper  entitled  "Discrimination  of  the  direction  and  speed  of 
motion  in  depth  from  binocular  information  alone"  by  C.  Portfors-Yeomans 
and  D.  Regan  is  in  press  in  the  Journal  of  Experimental  Psychology:  Human 
Perception  and  Performance,  and  the  findings  reported  to  ARVO  1995. 

The  broad  issue  here  is  collision  avoidance  and  in  particular  "will  it  hit 
me  or  miss  me?".  This  experiment  addresses  a  subject's  ability  to  judge  the 
direction  of  motion  in  depth  of  a  monocularly-visible  approaching  object 
using  binocular  cues  only. 

The  background  is  as  follows.  We  showed  theoretically  that  (Ref.  231), 
for  motion  within  the  plane  that  contains  the  eyes,  the  direction  ((3  in  Fig.  2) 
of  an  object's  motion  in  depth  given  by 


and 


P  « tan'^  [ 


P  « tan 


pI{(d<})/dt)R  +  (d<t)/dt)L}  i  , 

1  ™  1  ~  tan 

rWdt)  -I 

2D(d5/dt)  J 

l-DCdS/dt)  J 

/  I{[(d<t)/dt)R/(d(l>/dt)L]+l} 

\ 

'^2D{[(d<t)/dt)R/(d(t)/dt)L]-l}^ 

(2) 


(3) 
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Equations  (2)  and  (3)  indicate  that,  for  an  object  in  the  straight-ahead  position, 
there  are  two  mathematically-equivalent  binocular  cues  to  the  direction  of 
motion  in  depth,  namely  the  ratio  (d(l)/dt)/(d5/dt),  and  the  ratio 
(d(|)/ dt)R/(d(t)/ dt)L,  where  d5/ dt  is  the  rate  of  change  of  binocular  disparity,  and 

dcj)/ dt  is  the  angular  velocity  of  the  object's  binocularly-fused  retinal  image, 
(dcj)/  dt)jj  is  the  angular  speed  of  the  object's  retinal  image  in  the  right  eye,  and 
(d(|)/dt)L  in  the  left  eye.  For  motion  within  the  plane  perpendicular  to  the 
plane  that  contains  the  eyes  only  the  (d<|)/dt)/(d5/dt)  cue  is  available. 

Previous  research  had  been  restricted  to  motion  in  depth  within  the 
horizontal  meridian  where  the  two  cues  are  confounded.  Nothing  was 
known  about  binocular  discrimination  of  the  detection  of  motion  in  depth 
for  motion  within  the  vertical  meridian,  and  little  was  known  about  the 
relative  importance  of  these  two  cues  for  motion  within  the  horizontal 
meridian. 

Any  given  trial  consisted  of  two  presentations.  The  first  was  always  a 
reference  stimulus.  The  second  was  one  of  64  test  stimuli.  Each  of  the  64  test 
stimuli  comprised  a  different  combination  of  8  values  of  [(d(l)/dt)/(d5/dt)]  and 
8  values  of  speed.  Speed  and  direction  [i.e.  (d(t)/dt)/(d5/dt)]  were  orthogonal 
in  stimulus  space,  i.e.  there  was  zero  correlation  between  the  two  variables. 
The  speed  and  the  value  of  [(d<t)/ dt)/(d5/ dt)]  for  the  reference  stimulus  were 
equal  to  the  means  for  the  64  test  stimuli.  Subjects  were  instructed  signal  (1) 
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Figure  2  Geometry  of  binocularly-viewed  motion  in  depth.  LE,  RE:  left 
eye,  right  eye.  Nl,  Nr:  first  nodal  points  of  left  and  right  eyes  respectively. 
The  heavy  arrowed  line  represents  the  velocity  of  an  object  at  point  A 
moving  in  a  straight  line  at  constant  speed  V.  Point  B  is  midway  between  the 

eyes,  and  AC=D. 
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Figure  3  A-C:  The  percentage  of  "wider  of  the  head  than  the  reference 
trajectory"  responses  is  plotted  versus  the  task-relevant  variable 
(d(|)/dt)/(d5/dt)  in  A,  versus  the  task-irrelevant  variable  d5/dt  in  B,  and 
versus  the  task-irrelevant  variable  A5  in  C.  D-F:  The  percentage  of  "faster 
than  the  reference"  responses  is  plotted  versus  the  task-irrelevant  variable 
(d<|>/dt)/(d5/dt)  in  D,  versus  the  task-relevant  d5/dt  variable  in  E,  and  versus 
the  task-irrelevant  variable  A5  in  F.  G-I:  The  percentage  of  "excursion 
farther  than  the  reference"  responses  is  plotted  versus  the  task-irrelevant 
variable  (d(t)/dt)/(d5/dt)  in  G,  versus  the  task-irrelevant  variable  d5/dt  in  H 
and  versus  the  task-relevant  variable  A5  in  1.  All  three  variables  are 
expressed  as  fractional  departures  from  the  reference  value.  The  reference 
values  of  (d(t)/dt)/(d5/dt),  d5/dt  and  A5  were,  respectively,  0.6  deg,  0.4  deg/s 
and  11  min  arc  (1.0  on  the  abscissae). 
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REFERENCE  DIRECTION  (DEG) 

Figure  4  Discrimination  threshold  for  simulated  direction  (P)  are  plotted 
as  ordinate  versus  simulated  reference  direction  (Pref)-  Task-relevant 
thresholds  for  horizontal  and  vertical  meridians  are  shown,  respectively,  as 
m  and  X.  A:  Observer  1.  B:  Observer  2.  C:  Observer  3. 
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Figure  5  Discrimination  thresholds  for  simulated  speed  (S)  are  expressed 
as  a  percentage  of  the  reference  speed  (Sref)/  and  plotted  as  ordinate  versus 
simulated  reference  direction(PREF).  Task-relevant  thresholds  for  horizontal 
and  vertical  meridians  are  shown,  respectively,  as  m  and  X.  A:  Observer  1.  B: 

Observer  2.  C:  Observer  3. 
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whether  the  test  stimulus  would  pass  wider  of  the  head  than  the  reference 
and  (2)  whether  the  test  stimulus  was  faster  than  the  reference. 

Data  were  collected  for  36  different  reference  trajectories,  18  for  motion 
within  the  plane  coimtering  the  eyes  and  18  for  motion  perpendicular  to  that 
plane. 

In  a  control  experiment  we  required  observers  to  make  the  following 
three  discrimination  after  each  trial,  (a)  the  direction  of  motion;  (b)  the  speed 
of  motion;  (c)  the  disparity  displacement  (excursion).  We  used  a  set  of  216  test 
stimuli  comprising  6  values  of  direction  [i.e.  (d(t)/dt)/(d5/dt)],  6  values  of 
speed  (d5/dt)  and  6  values  of  disparity  excursion  (AS).  These  three  variables 
were  orthogonal  within  a  set  of  216  stimuli.  The  three  sets  of  responses  were 
each  analysed  along  three  stimulus  directions  giving  the  9  psychometric 
functions  shown  in  Fig.  3A-I.  When  the  task  was  to  discriminate  direction 
the  psychometric  function  was  steep  when  plotted  with  respect  to  direction 
(Fig.  3A)  but  flat  when  plotted  with  respect  to  the  other  two  variables.  We 
conclude  that  observers  ignored  speed  and  excursion  when  discriminating 
direction.  Fig.  3D-F  and  G-I  support  corresponding  conclusions  for  speed  and 
excursion  discriminations. 

We  used  stepwise  multiple  regression  to  show  that  when 
discriminating  the  direction  of  motion  in  depth,  observers  based  their 
discriminations  on  the  ratio  (d<j)/dt)/(d6/dt)  and  totally  ignored  trial-to-trial 
variations  in  the  frontal  plane  component  of  speed  (d<|)/dt)  and  in  (d5/dt)  and 
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in  displacement  A5  and  in  displacement  A<t)  and  in  the  speed  of  the  left  or 
right  eye's  retinal  image  alone.  We  used  stepwise  multiple  regression  to 
show  that,  when  discriminating  speed,  observers  based  their  discriminations 
on  dS/dt  and  totally  ignored  trial-to-trial  variations  in  d<t)/dt,  A5,  and  in  the 
speed  of  either  eye's  retinal  image  alone.  Direction  discrimination  thresholds 
and  speed  discrimination  thresholds  were  the  same  for  motion  within 
horizontal  and  vertical  meridians.  Fig.  4A  -  C  and  Fig.  5A  -  C  respectively 
show  how  direction  discrimination  thresholds  and  speed  discrimination 
thresholds  varied  with  reference  direction  (P  in  Fig.  2)  for  three  observers. 

3.4  Time  to  collision  judgements  in  foveal  and  peripheral  vision 

This  experiment  is  completed.  A  paper  entitled  "Visual  processing  of 
looming  and  time  to  contact  throughout  the  visual  field"  by  D.  Regan  &  A. 
Vincent  has  been  reported  to  ARVO  1994  and  published  in  Vision  Research 
(1995)  35, 1845-1857  (Ref.  235). 

The  broad  issue  is  collision  avoidance  in  real  aircraft.  This  experiment 
addresses  the  question  of  a  subject's  ability  to  judge  time  to  collision  when  the 
subject  does  not  gaze  directly  at  the  approaching  object,  but  rather  views  the 
object  in  peripheral  vision.  No  previous  information  was  available  on  this 
question  because  all  previous  studies  were  restricted  to  foveal  vision. 

First,  we  addressed  the  point  that  the  rate  of  expansion  of  an  object's 
retinal  image  does  not  indicate  time  to  collision.  For  example,  deg/sec  rate  of 
retinal  image  expansion  for  an  object  20ft  wide  is  twice  the  rate  of  expansion 
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for  an  object  10ft  wide,  even  though  they  are  at  the  same  distance  and 

approaching  at  the  same  speed  with  the  same  time  to  collision.  To  judge  time 

to  collision  (T)  for  an  object  of  unknown  size  on  the  basis  of  equation  (1) 

above  [i.e.  T  «  6/ (d9/dt)],  a  subject  must  respond  to  the  ratio  6/(d9/dt) 

independently  of  variations  in  retinal  image  size  (9)  and  rate  of  expansion 

(d9/ dt).  We  showed  that  this  can  be  done  using  the  following  procedure. 

An  approaching  object  was  simulated  by  generating  a  square  on  a 

monitor  using  electronics  of  our  own  design  that  have  been  described 

elsewhere  (Ref.  220).  The  size  of  the  square  varied  according  to  the  equation 

tan  9o 

tan9t=  (4) 

where  29t  was  the  square's  angular  subtense  at  time  t,  and  29o  &  Tq  were, 

respectively,  the  square's  angular  subtense  and  time  to  collision  at  time  t=0. 

Equation  (4)  is  depicted  graphically  in  Fig.  6. 

Experiment  1 

Psychophysical  methods 

The  rationale  of  the  psychophysical  method  can  be  understood  by 
reference  to  Fig.  6.  The  three  curves  are  for  times  to  contact  of  To,  2To  and  3To 
respectively,  and  the  starting  diameter  (i.e.,  29o)  is  arbitrarily  set  at  2  deg. 

Suppose  that  we  presented  a  subject  with  several  values  of  time  to 
contact  as  illustrated  in  Fig.  6,  four  of  which  were  shorter  than  2.0  sec  and  four 
of  which  were  longer  than  2.0  sec,  and  measured  the  just-noticeable  difference 
from  2.0  sec.  We  would  be  unable  to  conclude  that  the  discrimination 
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STARTING  SIZE  CONSTANT 

TIME  TO  CONTACT - ► 

RATE  OF  EXPANSION  - ► 


Figure  6  The  angular  size  of  the  retinal  image  of  a  rigid  spherical  object 
that  is  moving  at  constant  speed  along  the  line  of  sight  (ordinate)  is  plotted 
versus  time  (abscissa).  Curves  are  shown  for  three  times  to  contact:  Tq,  2To 
and  3To.  The  dotted  lines  illustrate  that  the  initial  rate  of  expansion  of  the 
retinal  image  is  inversely  proportional  to  the  time  to  contact.  Starting  size  is 
arbitrarily  taken  to  be  2  deg. 
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'nUE  TO  CONTACT 


■ 


RATK  OF  EXPANSION 
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TTCANOROS 


tTARTOiaaiZZ 


Figure  7  A-D:  Predicted  response  patterns  for  (A)  discriminations  of  time 
to  contact  (TTC),  (B)  discriminations  of  rate  of  expansion  (ROE),  (C) 
discriminations  of  TTC  assuming  that  variations  of  TTC  produce  illusory 
variations  in  perceived  ROE  so  that  variations  in  ROE  and  TTC  are 
completely  confounded  and  (D)  discriminations  of  starting  size  (SS). 
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Figure  S  A,  C:  Empirical  response  patterns  for  discriminating  time  to 
contact  (TTC)  at  0  deg  and  32  deg  eccentricity  respectively.  B,  D:  Empirical 
response  patterns  for  discriminating  rate  of  expansion  (ROE)  at  0  deg  and  32 
deg  eccentricity  respectively. 


■  SOONER  THAN  THE  MEAN  TIME 
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Figure  9  Two  psychometric  functions  for  the  same  task.  The  percentage 
of  1.0  on  the  abscissa  corresponds  to  a  2.0  sec  time  to  contact .  In  B  &  D,  1.0  on 
the  abscissa  corresponds  to  an  initial  rate  of  expansion  of  0.17  deg/sec. 


Department  of  Psychology 
FAX:  (416)  736-5814 


UNIVERSITE 


FACULTY  OF  ARTS 

4700  KEELE  STREET  .  NORTH  YORK  •  ONTARIO  •  CANADA  •  M3J  1P3 


ECCENTRICITY  (DEG) 


Figure  10  Discrimination  threshold  (ordinate)  versus  visual  field 
eccentricity  for  vertical  and  horizontal  meridia.  The  observer's  task  was  to 
judge  whether  the  time  to  contact  of  any  given  stimulus  was  sooner  or  later 
than  the  mean  of  the  set  of  64  stimuli.  Open  circles  indicate  thresholds 
calculated  with  respect  to  0o/(d0/dt)o.  Filled  squares  indicate  thresholds 
calculated  with  respect  to  (d0/dt)Q.  The  hatched  area  in  D  marks  the  blind 
spot.  A:  Upper  field.  B:  Lower  field.  C:  Left  field.  D:  Right  field. 
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threshold  measured  was  for  time  to  contact,  because  the  stimulus  illustrated 
in  Fig.  6  completely  confounds  time  to  contact  with  rate  of  expansion.  The 
dotted  lines  in  Fig.  6  show  that  the  initial  rate  of  expansion,  for  example, 
decreased  according  to  the  progression  3;  2;  1  while  time  to  contact  increased 
according  to  the  progression  1;  2;  3.  The  basic  idea  is  as  follows.  We  use  a 
segment  of  the  graphs  in  Fig.  6A-C  between  t  =  0  and,  at  most,  two-thirds  of 
the  time  to  contact.  Suppose  now  that  the  ordinates  in  these  three  segments 
are  multiplied  by  1.0,  2.0  and  3.0  respectively.  The  initial  rate  of  expansion  (i. 
e.,  the  slope  of  the  dotted  line)  will  now  be  the  same  for  the  three  times  to 
contact,  and  the  rates  of  expansion  will  be  approximately  equal  at 
corresponding  values  along  the  three  segments  for  the  three  times  to  contact. 
Now  suppose  we  construct  a  second  and  a  third  horizontal  row  by 
multiplying  the  ordinate  in  the  three  new  graph  segments  by  factors  of  2  and 
3  respectively.  We  now  have  a  3  x  3  array  of  stimuli  in  which  time  to  contact 
varies  horizontally,  rate  of  expansion  varies  vertically,  and  starting  size  varies 
both  horizontally  and  vertically. 

In  the  present  study  we  used  an  8  x  8  rather  than  a  3  x  3  array  of  stimuli. 
In  addition,  rather  than  a  1;  2;  3  progression,  time  to  contact  varied 
horizontally  according  to  the  following  progression:  n'^  ®;  n'^-^S;  n"®-®;  n'^-^S; 
nO.25^  nO  75.  j^i.o  (The  value  of  n  set  the  difficulty  of  the  task.  We  set  n  = 
1.3).  Initial  rate  of  expansion  varied  vertically  according  to  the  same 
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progression.  Our  purpose  in  structuring  the  array  in  this  way  was  to  create  a 
diagonal  symmetry  for  starting  size  (see  Fig.  7D). 

The  subject  was  provided  with  two  buttons  in  the  first  two  parts  of 
Experiment  1.  In  part  1,  the  subject  was  instructed  to  press  button  1  or  2 
depending  on  whether  time  to  contact  was  sooner  or  later  than  the  mean  of 
the  set  of  64  stimuli.  In  part  2,  the  subject  was  instructed  to  press  button  1  or  2 
depending  on  whether  the  rate  of  expansion  was  faster  or  slower  than  the 
mean  for  the  set  of  64  stimuli.  Auditory  feedback  was  provided.  The  subject's 
responses  were  stored  in  an  8  x  8  response  array,  which  corresponded  to  the  8 
X  8  stimulus  array.  In  part  1,  the  response  array  was  for  time  to  contact 
judgements,  and  in  part  2  for  rate  of  expansion  judgements.  Each  of  the  64 
cells  in  each  of  the  two  arrays  had  16  repeats  (i.e.  2,048  responses  in  all).  Data 
were  collected  at  eccentricities  of  0  deg  and  32  deg  in  the  left  visual  field. 
Response  prediction  and  data  analysis 

Fig.  7A-D  illustrate  four  patterns  of  response  predicted  from  the 
structure  of  the  stimulus  array.  In  Fig.  7 A  we  assume  that  the  subject  bases 
respor\ses  entirely  on  trial-to-trial  variations  of  time  to  contact  and  ignores 
trial-to-trial  variations  in  rate  of  expansion  and  starting  size.  In  Fig.  7B  we 
assume  that  the  subject  bases  responses  entirely  on  trial-to-trial  variations  in 
rate  of  expansion,  and  ignores  trial-to-trial  variations  in  time  to  contact  and 
starting  size.  In  Fig.  7C  we  assume  that  the  subject  confoimds  trial-to-trial 
variations  of  time  to  contact  with  trial-to-trial  variations  in  rate  of  expansion. 
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but  ignores  trial-to-trial  variations  in  starting  size.  Trial-to-trial  variations  in 
time  to  contact  and  trial-to-trial  variations  in  rate  of  expansion  were 
combined  by  linear  summation  with  a  relative  weighting  of  21:1  (see  below 
for  the  reason  for  choosing  this  weighting).  In  Fig.  7D  we  assume  that  the 
subject  bases  responses  entirely  on  trial-to-trial  variations  in  starting  size,  and 
ignores  trial-to-trial  variations  in  time  to  contact  and  rate  of  expansion.  The 
major  difference  between  the  predictions  shown  in  Fig.  7C  and  D  is  that  the 
100%  response  prediction  is  for  the  largest  rate  of  expansion  and  shortest  time 
to  contact  in  Fig.  7C  but  for  the  largest  rate  of  expansion  and  longest  time  to 
contact  in  Fig.  7C.  This  difference  is  a  consequence  of  assuming  that  an 
increase  in  rate  of  expansion  produces  an  illusory  decrease  in  time  to  contact. 

Data  obtained  in  Experiment  1  were  analyzed  quantitatively  as  follows. 
"Sooner  than  the  mean  time  to  contact"  responses  in  each  of  the  vertical 
columns  of  the  time  to  contact  8x8  response  array  were  summed  and  plotted 
versus  the  initial  time  to  contact  6q/ (d6/dt)g  to  give  a  psychometric  fvmction. 

Then  the  same  response  array  was  re-analysed  by  summing  each  of  the 
horizontal  rows,  and  plotted  versus  the  iiutial  rate  of  expansion  (de/dt)o  to 

give  a  second  psychometric  fimction.  Discrimination  threshold  with  respect 
to  time  to  contact  was  defined  as  (l/2){[eo/(de/dt)o]75  -  [eo/(de/dt)o]25},  where 
[0o/(de/dt)o]75  and  [0o/(de/dt)o]25  were,  respectively,  the  values  of  0o/(d0/dt)o 
for  75%  and  25%  "sooner  than  the  mean  time  to  contact"  responses. 
Discrimination  threshold  with  respect  to  rate  of  expansion  was  defined 
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analogously.  Discrimination  thresholds  were  estimated  from  the 
psychometric  functions  by  Probit  analysis. 

Results 

Comparison  of  predicted  response  patterns  with  foveal  and  peripheral  data 

In  Fig.  8A  the  subject  was  instructed  to  judge  whether  the  time  to 
contact  of  any  given  stimulus  was  sooner  or  later  than  the  mean  time  to 
contact  of  the  set  of  64  stimuli  (part  1  of  Experiment  1).  Stimulus  eccentricity 
was  0  deg.  The  pattern  of  responses  was  close  to  the  Fig.  7 A  prediction  and 
quite  different  from  any  of  the  Fig.  7B-D  predictions.  We  conclude  that,  in 
foveal  vision,  the  subject  based  responses  on  trial-to-trial  variations  of  time  to 
contact  and  ignored  trial-to-trial  variations  in  rate  of  expansion  and  starting 
size. 

In  Fig.  8B  the  stimulus  set  was  exactly  the  same  as  in  Fig.  8A.  Only  the 
subject's  instructions  were  different.  In  Fig.  8B  the  subject  was  instructed  to 
judge  whether  the  rate  of  expansion  of  any  given  stimulus  was  greater  or 
smaller  than  the  mean  rate  of  expansion  of  the  set  of  64  stimuli  (part  2  of 
Experiment  1).  Stimulus  eccentricity  was  0  deg.  The  pattern  of  responses  was 
close  to  the  Fig.  7B  prediction,  and  quite  different  from  any  of  the  Fig.  7A,  C  or 
D  predictions.  We  conclude  that,  in  foveal  vision,  the  subject  based  responses 
on  trial-to-trial  variations  in  rate  of  expansion,  and  ignored  trial-to-trial 
variations  in  time  to  contact  and  starting  size. 
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Compared  with  0  deg  eccentricity,  at  32  deg  eccentricity  the  subject  was 
less  able  to  base  his  judgements  entirely  on  trial-to-trial;  variations  in  time  to 
contact  (compare  Fig.  8C  with  Fig.  7 A).  The  similarity  between  the  Fig.  8C 
response  pattern  and  the  Fig.  7C  prediction  is  consistent  with  the  hypothesis 
that  an  x%  variation  in  rate  of  expansion  produced  an  illusory  variation  in 
time  to  contact  of  roughly  (x/2)°/°,  and  the  subject  ignored  trial-to-trial 
variations  in  starting  size.  Additional  confirmation  of  this  point  is  provided 
by  the  considerable  difference  between  the  Fig.  8C  response  pattern  and  the 
Fig.  7D  prediction. 

Comparison  of  discrimination  thresholds  in  the  fovea  and  periphery 

The  two  psychometric  functions  in  Fig.  9A  &  B  were  obtained  from  the 
Fig.  8A  response  matrix.  When  the  percent  of  "sooner"  responses  was  plotted 
vs  the  ratio  Qq/ (d0/dt)o,  a  much  steeper  psychometric  function  resulted  than 
when  the  same  responses  were  plotted  vs  (d0/dt)Q.  [(Discrimination 
threshold  was  6.7  %  (SE  =  0.4)  in  Fig.  9A.  In  Fig.  9B  threshold  was  so  high  that 
it  was  not  possible  to  be  more  precise  than  to  say  that  it  was  greater  than 
100%].  Fig.  9C  &  D,  were  obtained  from  the  9C  response  matrix,  and  show 
that  at  an  eccentricity  of  32  deg  the  difference  between  the  two  psychometric 
functions  was  considerably  less  than  in  foveal  vision.  Although  the  subject's 
responses  were  still  predominantly  determined  by  the  ratio  0o/(d0/dt)o, 

discrimination  threshold  in  Fig.  9C  was  only  21  times  lower  than  in  Fig.  9D 
(21%  compared  with  44%). 
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Experiment  2 
Psychophysical  methods 

In  Experiment  2,  data  were  collected  for  judgements  of  time  to  contact 
and  rate  of  expansion  at  20  locations  in  the  visual  field  between  0  deg  and  32 
deg  eccentricity  along  the  horizontal  and  vertical  meridia. 

Results 

Effect  of  eccentricity  on  the  ability  to  discriminate  time  to  contact. 

The  data  shown  in  Fig.  10  were  obtained  by  instructing  subject  1  to 
judge  whether  the  time  to  contact  for  any  given  stimulus  was  sooner  than  or 
later  than  the  mean  for  the  set  of  64  stimuli.  The  thresholds  plotted  in  Fig.  10 
were  estimated  from  psychometric  fimctions  of  the  kind  shown  in  Fig.  9. 
Open  circles  (continuous  lines)  were  obtained  by  plotting  the  subject's 
response  vs  the  task-relevant  variable,  and  filled  squares  (dashed  lines)  were 
obtained  by  plotting  the  subject's  responses  vs  the  initial  rate  of  expansion  (a 
task-irrelevant  variable). 

Conclusions 

Our  main  conclusion  concerns  the  ability  to  discriminate  between 
different  times  to  contact  with  an  approaching  object  independently  of  the 
size  and  rate  of  expansion  of  the  object's  retinal  image.  This  ability  is  high 
within  the  central  visual  field  (within  about  4  to  8  deg  eccentricity  depending 
on  the  subject  and  meridian).  Our  results  suggest  that,  outside  this  central 
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visual  field  judgements  of  an  object's  time  to  contact  would  no  longer  be 
independent  of  the  object's  size. 

The  qualitative  agreement  between  the  Fig.  8C  data  and  the  Fig.  7C 
prediction  is  consistent  with  the  hypothesis  that,  in  peripheral  vision,  trial-to- 
trial  variations  in  rate  of  expansion  produced  an  illusory  percept  of  trial-to- 
trial  variatioris  in  time  to  contact.  We  conclude  that  illusory  percepts  are  an 
important  reason  why,  for  time  to  contact  discrimination,  the  separation 
between  the  task-relevant  and  task-irrelevant  thresholds  in  Fig.  10  decreases 
with  increasing  eccentricity. 

Relevance  for  Aviation 

The  relevance  for  collision  avoidance  in  aviation  is  as  follows.  It  is 
natural  that  to  judge  time  to  collision  a  pilot  should  look  directly  at  an 
oncoming  object  on  a  collision  course.  We  report  that  that  is  the  correct  thing 
to  do.  However,  it  is  impossible  to  look  directly  at  two  things  at  once.  It  is  not 
intuitively  obvious  that  judgements  of  time  to  contact  are  unreliable  in 
peripheral  vision.  Our  results  imply  that  pilots  should  be  made  aware  of  this 
deficiency  in  the  human  visual  system  so  that,  when  confronted  by  a  collision 
hazard  with  two  or  more  oncoming  objects  the  pilot  can  adopt  some 
deliberate  strategy  such  as,  for  example,  glancing  rapidly  from  one  to  the 
other. 

3.5  Time  to  collision  judgements:  effects  of  mismatching  rates  of 
expansion  of  size  and  texture  elements. 
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A  paper  is  in  press  in  Perception  and  Psychophysics  (Ref.  25S) 

The  broad  issue  is  flight  simulator  training  for  collision  avoidance. 
This  experiment  addresses  the  question  of  whether  a  subject's  ability  to  judge 
the  time  to  collision  with  a  simulated  oncoming  object  is  degraded  by  a 
mismatch  between  the  rates  of  expansion  of  the  object  and  its  surface  texture. 
The  relevance  of  the  study  is  that  many  flight  simulators  generate  object 
texture,  but  in  not  all  cases  are  the  texture  and  object  dynamics  matched. 

The  background  is  our  previous  (1983)  laboratory  study  in  which  we 
investigated  the  perception  of  motion-in-depth  produced  by  an  expanding 
image  texture.  We  found  that  if  image  size  and  texture  were  not  matched 
accurately,  the  motion  in  depth  aftereffect  could  be  weakened  or  even 
abolished  (Ref.  125). 

Apparatus  and  methods 

We  designed  and  built  hardware  electronics  that  generates  a  square 
covered  with  sharp-edged  rectangular  texture  elements  on  a  monitor  (Fig.  11). 
The  expansion  of  the  square  and  the  expansion  of  the  texture  element  can  be 
controlled  independently  so  as  to  vary  the  ratio  R,  where  R  =  TE/SE  (TE  and 
SE  are,  respectively,  the  %  expansion  per  sec  of  texture  elements  and  the 
square).  The  ratio  R  can  be  varied  from  0.0  (texture  size  constant,  square 
expands)  through  1.0  (texture  size  and  square  size  perfectly  locked)  to  2.0 
(texture  expands  twice  as  fast  as  the  square). 
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Figure  11  Photographs  of  three  of  the  stimulus  displays.  The  conditions 
shown  are  R  =  2.0,  R  =  1.0  and  R  =  0.0  in  A,  B  &  C  respectively  where  R  = 
(relative  rate  of  expansion  of  the  texture  elements  within  the 
square)/ (relative  rate  of  expansion  of  the  square). 
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Figure  12  Estimated  time  to  collision  in  msec  is  plotted  as  ordinate  versus 
the  ratio  R  where  R  =  (relative  rate  of  expansion  of  the  texture  elements 
within  the  square)/ (relative  rate  of  expansion  of  the  square).  Filled  circles 
and  open  triangles  are  for  two  observers.  The  vertical  bars  indicate  ±1 
standard  error. 
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The  psychophysical  procedure  is  the  same  as  just  described  in  Section 
3.4  above.  Subjects  were  presented  with  64  stimuli,  each  of  which  comprised 
a  different  combination  of  time  to  collision,  square  starting  size  and  square 
rate  of  expansion.  The  ratio  TE/SE  was  fixed.  Subjects  were  instructed  to 
judge  whether  time  to  contact  was  sooner  or  later  than  the  mean  of  the  set  of 
64  stimuli.  Data  were  analysed  as  in  Section  3.4  above  so  as  to  reveal  the 
degree  to  which  subjects  unconfounded  trial-to-trial  variations  in  time  to 
contact  and  trial-to-trial  variations  in  starting  size  and  in  rate  of  expansion. 
The  experiment  was  then  repeated  for  several  values  of  the  ratio  TE/SE. 
Results 

Our  main  finding,  illustrated  in  Fig.  11,  is  that  the  ability  to  ignore 
trial-to-trial  variations  in  rate  of  expansion  when  discriminating  time  to 
contact  is  excellent  when  texture  size  and  square  size  are  locked  (TE/SE  =  1.0), 
but  is  poor  when  texture  expands  more  slowly  than  the  square.  In  addition, 
discrimination  threshold  rises  as  the  ratio  TE/SE  is  reduced. 

3.6  Cyclopean  processing  of  changing-disparity  and  changing-size. 

Results  have  been  reported  to  ARVO  1995  and  a  report  "Cyclopean 
motion  perception  produced  by  oscillations  of  size,  disparity  and  location"  by 
R.  Gray  and  D.  Regan  published  in  Vision  Research  (1996)  35,655-666. 

The  broad  issue  is  the  perception  of  motion-in-depth  produced  by 
either  binocular  or  monocular  cues  (specifically,  changing-disparity  and 
changing-size).  This  experiment  is  designed  to  distinguish  between 
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processing  that  occurs  before  and  after  the  signals  from  left  and  right  eyes 
unite. 

Apparatus  and  methods 

A  stereopair  of  random  dots  was  generated  on  a  pair  of  monitors  by  our 
own  hardware  electronics,  and  presented  to  the  subject  dichoptically.  In 
binocular  view  the  dot  pattern  contained  a  square  standing  out  in  depth.  In 
our  present  experiment  a  different  pair  of  random  dot  patterns  was  generated 
50  times  per  sec  so  that  in  binocular  view  the  square  and  the  background  were 
filled  with  twinkling  dots  (the  so-called  dynamic  random  noise  display).  We 
used  dynamic  random  noise  so  that,  even  when  it  moved,  the  square  could 
not  be  seen  when  one  eye  was  closed.  The  square  just  described  was 
Cyclopean.  A  monocularly-visible  square  was  created  by  using  a  static  random 
dot  pattern  in  the  surround  while  leaving  the  square  filled  with  dynamic 
random  noise  just  as  in  the  cyclopean  case.  The  subject's  ocular  convergence 
was  controlled  by  fixating  on  nonious  lines  in  the  plane  of  the  background. 

The  method  of  constant  stimuli  was  used  in  Expt  1.  The  square's 
disparity  was  oscillated  by  a  wavetrain  of  5  sinewave  cycles  windowed  by  a 
Gaussian.  Each  trial  comprised  a  reference  presentation  (oscillation 
amplitude  zero)  and  a  test  presentation  with  oscillation  amplitude  set  to  one 
of  5  values.  The  task  was  to  decide  which  presentation  contained  the  depth 
oscillation.  Oscillation  thresholds  were  measured  over  a  frequency  range  of 
0.25  to  8  Hz  for  the  cyclopean  square. 
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Figure  13  A:  Peak-to-peak  amplitude  of  disparity  oscillation  at  threshold 
for  motion  in  depth  perception  (ordinate)  is  plotted  versus  oscillation 
frequency.  Filled  and  ojjen  symbols  are  for  cyclopean  and  monocularly- 
visible  targets  respectively.  B,  C:  Corresponding  data  for  two  other  observers. 
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Figure  14  A:  Peak-to-peak  amplitude  of  size  oscillations  at  threshold  for 
motion  in  depth  perception  (ordinate)  is  plotted  versus  oscillation  frequency. 
Filled  and  open  symbols  are  for  cyclopean  and  monocularly-visible  targets 
respectively.  C,  E:  Corresponding  data  for  two  other  observers.  B,  D,  F:  As  for 
A,  C  &  E  except  that  observers  set  for  threshold  of  size  oscillation  perception. 
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In  Fig.  13,  filled  circles  (continuous  lines)  plot  motion-in-depth  thresholds 
for  the  Cyclopean  square.  Open  squares  (dashed  lines)  plot  corresponding  data 
for  the  monocularly-visible  square.  The  similarity  of  cyclopean  and 
noncyclopean  data  lead  us  to  conclude  that  the  shape  of  the  curve  is 
determined  by  completely  before  signals  from  the  two  eyes  have  united,  i.e. 
attenuation  at  cyclopean  level  is  negligible. 

In  Expt  2  the  size  of  the  square  rather  than  its  disparity  was  oscillated. 
The  size  oscillation  produced  the  impression  that  the  square  was  moving  in 
depth,  but  the  illusory  depth  movement  was  much  stronger  for  the 
noncyclopean  square  than  for  the  cyclopean  square.  Filled  and  open  circles  in 
Fig.  14A,  C  &  E  plot  depth  oscillation  thresholds  for  the  cyclopean  and 
monocularly-visible  squares  respectively.  In  contrast  with  the  corresponding 
data  for  motion-in-depth  sensation  produced  by  disparity  oscillations  (Fig.  13), 
the  cyclopean  and  noncyclopean  thresholds  are  quite  different,  indicating  that 
a  cyclopean  changing-size  stimulus  is  much  less  effective  than  a 
monocularly-visible  changing  size  stimulus. 

This  finding  was  confirmed  in  Expt  3  where  we  pitted  size  oscillations 
versus  simultaneous  oscillations  of  disparity.  Subjects  were  instructed  to 
adjust  the  disparity  oscillation  amplitude  so  as  to  cancel  the  sensation  of 
depth  oscillations  produced  by  a  fixed  amplitude  of  size  oscillation. 

3.7  Shape  discrimination  for  cyclopean  form 
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A  report  has  been  published.  D.  Regan  &  S.  J.  Hamstra  "Shape 
discrimination  for  rectangles  defined  by  disparity  alone,  disparity  plus 
luminance  and  by  disparity  plus  motion.  Vision  Research.  34, 2277-2291 
(1994). 

The  broad  issue  is  the  role  of  binocular  vision  in  seeing  and 
recognizing  objects.  The  work  is  relevant  to  (a)  the  design  of  stereo  displays 
viewed  by  the  operators  of  remotely-controlled  vehicles  and  (b)  collision 
avoidance  in  nap-of-the-Earth  flight.  The  experiment  addresses  a  subject's 
ability  to  discriminate  a  change  in  the  shape  (aspect  ratio)  of  a  rectangle  that  is 
visible  in  binocular  vision  but  invisible  in  monocular  vision,  i.e.  a  cyclopean 
rectangle. 

Methods 

A  pseudo-random  pattern  of  bright  dots  was  generated  by  shift  registers 
in  laboratory-designed  and  built  hardware  electronics  (Ref.130)  and  displayed 
on  electrostatically-controlled  monitors.  As  illustrated  in  Fig.  15 A,  the  two 
monitors  (Ml  &  M2)  were  viewed  dichoptically.  At  the  viewing  distance  of 
120  cm,  the  dot  pattern  was  a  4  x  4  deg  square.  Each  dot  subtended  about  2.0 
min  arc.  There  were  approximately  770  dots  on  each  monitor  so  that  dot 
density  was  approximately  48  per  deg2.  By  means  of  pellicles  PI  and  P2, 
nonious  lines  were  optically  superimposed  on  the  dot  displays.  The  nonious 
lines  were  created  by  back-illuminating  by  means  of  light-emitting  diodes  two 
slits  SI  and  S2. 
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Figure  15  A:  Optical  arrangement.  Ml  and  M2  are  monitors.  P1-P5  are 
pellicles.  Si  and  S2  are  illuminated  slits.  PLD  is  an  illuminated  plane  of  large 
dots.  XY:  XY  plotter  with  the  pen  replaced  by  an  illuminated  bar  (D).  B: 
Stimulus  field  from  observer's  viewpoint  showing  the  square  central 
dichoptically-viewed  surround  pattern  of  fine  dots  generated  by  the  monitors 
surrounded  by  the  binocularly-viewed  surround  plane  of  large  dots.  Nonious 
lines  are  shown  at  the  centre  of  the  pattern  of  fine  dots.  C:  View  from  above 
the  subject.  For  clarity,  the  pattern  of  fine  dots  is  enlarged.  The  PLD  plane  has 
crossed  disparity  and  the  nonious  lines  (NL)  have  been  adjusted  to  place 
them  in  the  plane  of  the  stimulus  rectangle.  D:  As  for  C,  but  the  PLD  plane 
has  uncrossed  disparity. 
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Figyirg  16  Stereograms  to  illustrate  a  disparity-defined  rectangle  with 
nonious  lines.  View  the  stereogram  from  arm's  length.  Hold  a  pencil  in 
front  of  the  page  at  such  a  distance  that  with  the  left  eye  closed  the  point  is  at 
the  centre  of  the  left  dot  pattern,  and  with  the  right  eye  closed  the  point  is  at 
the  centre  of  the  right  dot  pattern.  Now  open  both  eyes  and  look  at  the  pencil 
point.  After  a  few  moments  you  will  see  three  rather  than  two  dot  patterns. 
The  central  pattern  is  binocularly-fused  and  the  camouflaged  rectangle  will 
emerge.  The  pencil  can  now  be  removed. 
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DISPARITY  (min  aiv) 

Figure  17  Aspect  ratio  discrimination  threshold  for  a  disparity-defined 
rectangle  is  plotted  as  ordinate  versus  the  disparity  of  the  surround.  The  y 
axis  is  linear  and  the  x  axis  is  logarithmic.  Vertical  bars  through  the  data 
points  indicate  1  standard  error.  For  disparities  below  about  0.1  min  arc  the 
visibility  of  the  rectangle  was  so  low  that  the  discrimination  task  could  not  be 
carried  out.  A,  observer  1.  B,  observer  2.  C,  observer  3.  D,  observer  4. 
Vertical  arrows  indicate  disparity  thresholds  for  detecting  the  rectangle. 
Vertical  dashed  lines  demarcate  disparity  ranges  where  all  dots  were  fused 
(AF),  where  only  the  rectangle  was  fused  (ROF)  and  where  the  rectangle  was 
diplopic  (RD). 
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Figure  18  A:  An  observer's  left  (LE)  and  right  (RE)  eyes  view  a  far  plane  (F) 
through  a  hole  of  width  /i  in  a  near  plane  (N).  The  dashed  lines  pass  through 
the  edges  of  the  hole,  and  are  at  right  angles  to  the  two  planes.  B;  The 
observer's  left  and  right  eyes  subtend  a^  radians  at  the  near  plane  and  a2 
radians  at  the  far  plane.  C:  An  observer's  left  and  right  eyes  view  a  restricted 
region  of  a  near  plane  (N)  that  occludes  part  of  a  far  plane  (F). 
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Figure  19  A  &  B:  Solid  circles  plot  orientation  discrimination  threshold 
(ordinate)  vs  disparity  for  a  constant-width  bar.  The  vertical  arrows  indicate 
threshold  for  bar  detection. 
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Figure  20  Matched  depth  (ordinate)  vs  disparity  of  a  cyclopean  bar  for 
observer  1  (A,  C)  and  2  (B,  D).  The  fine  continuous  lines  indicate  zero 
disparity  at  the  viewing  distance  of  120  cm.  The  vertical  dashed  lines  indicate 
the  critical  disparity.  A  and  B  are  for  a  constant-width  bar,  and  C  and  D  for  a 
variable-width  bar. 
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The  dichoptically-viewed  4.0  x  4.0  deg  pattern  of  small  dots  was 
superimposed  on  a  4.0  x  4.0  deg  square  black  area  located  at  the  centre  of  a 
binocularly-viewed  uniformly-illuminated  plane  (PLD)  of  randomly  scattered 
large  circular  black  dots.  Fig.  15B  shows  what  the  subject  saw;  the 
dichoptically-viewed  stereogram  of  small  dots  with  superimposed  nonious 
lines,  all  surroimded  by  the  plane  of  large  dots.  Fig.  15C  &  D  are  views  from 
above,  showing  enlarged  views  of  the  rectangle  with  the  nonious  lines 
adjusted  to  be  in  the  same  plane  as  the  rectangle.  The  small  dots  surrounding 
the  rectangle  were  always  in  the  same  plane  as  the  plane  of  large  dots  (PLD). 

The  Cyclopean  rectangle  is  illustrated  in  Fig.  16. 

Each  trial  consisted  of  a  single  1.5  sec  presentation  of  the  rectangle.  All 
dots  on  both  monitors  were  switched  off  except  during  a  trial.  The 
surrounding  plane  of  large  dots  was  illuminated  throughout.  The  subject 
was  instructed  to  press  one  of  two  buttons  to  signify  whether  the  rectangle 
was  more  elongated  in  the  horizontal  or  in  the  vertical  direction.  Ten 
different  aspect  ratios  were  presented  in  random  order,  and  in  any  given  rim 
each  aspect  ratio  was  repeated  6  times.  A  run  lasted  about  5  mins.  Auditory 
feedback  was  provided. 

Results 

First  we  describe  the  subjective  appearance  of  the  stimulus.  When  the 
disparity  of  the  DD  rectangle  was  zero,  the  rectangle  was  perfectly 
camouflaged  even  when  viewed  dichoptically,  so  that  the  aspect  ratio 
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discrimination  task  was  impossible.  For  crossed  disparities  substantially 
above  detection  threshold  (arrowed  in  Fig.  17 A)  the  rectangle  was  seen  quite 
clearly,  standing  out  in  front  of  the  surround  dots,  all  dots  in  the  rectangle 
and  surround  being  fused  (AF  in  Fig.  17A  "crossed").  At  higher  disparities, 
fusion  was  lost  for  the  surround  dots,  though  the  rectangle  itself  remained 
fused  (ROF  in  Fig.  17A  "crossed").  For  uncrossed  disparities  substantially 
above  detection  threshold,  the  rectangle  was  seen  quite  clearly  through  what 
appeared  to  be  a  hole  in  the  surround  dots,  all  dots  in  the  rectangle  and 
surroimd  being  fused  (AF  in  Fig.  17A  "uncrossed").  Fusion  was  lost  for  the 
surround  dots  at  higher  disparities,  though  the  rectangle  itself  remained 
fused  (ROF  in  Fig.  17A  "uncrossed").  At  still  higher  disparities  the  rectangle 
became  diplopic  (RD  in  Fig.  17A  "uncrossed"). 

Although  the  best  values  for  crossed  and  uncrossed  disparities  were 
similar  (4.0  ±  0.5%  for  crossed  and  4.3  ±  0.5%,  for  uncrossed),  results  for 
crossed  and  uncrossed  disparities  differed  in  that  discrimination  threshold 
remained  low  over  a  comparatively  broad  range  of  crossed  disparities  (e.g.  for 
subject  1,  threshold  changed  from  4.0  ±  0.5  min  arc  to  6.3  ±  0.8  min  arc  as 
disparity  increased  from  1.0  to  126  min  arc)  while,  for  uncrossed  disparities, 
threshold  rose  steeply  on  either  side  of  the  minimum. 

Conclusions 

We  report  psychophysical  evidence  that  the  human  visual  pathway 
contains  a  mecharusm  sensitive  to  the  ratio  between  the  height  and  width  of 
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a  disparity-defined  rectangle.  On  the  grounds  that  we  jittered  the  rectangle's 
area  randomly  we  conclude  that  this  ratio-tuned  mechanism  operates  rather 
independently  of  the  rectangle's  linear  dimensions. 

Further  conclusions  are  as  follows.  First,  the  effect  of  disparity  on 
aspect  ratio  was  different  for  crossed  and  uncrossed  disparities  (Fig.  17). 
Second,  the  lowest  aspect  ratio  discrimination  threshold  was  approximately 
the  same  for  crossed  and  uncrossed  disparities.  For  both  crossed  and 
uncrossed  disparities,  aspect  ratio  discrimination  threshold  was  at  its  lowest 
value  at  or  just  before  the  maximum  disparity  at  which  both  rectangle  and 
surround  dots  were  still  fused,  and  beyond  this  point  discrimination 
threshold  increased. 

3.8  Orientation  discrimination  for  cvclopean  form 
Results  were  reported  to  ARVO  in  1994  and  a  paper  has  been  published.  S.  J. 
Hamstra  &  D.  Regan  (1994).  "Orientation  discrimination  in  cyclopean  vision" 
Vision  Research.  35, 365-374. 

The  broad  issue  is  the  same  as  in  Section  3.7.  This  experiment 
addresses  a  subject's  ability  to  discriminate  the  orientation  of  a  bar  that  is 
visible  in  binocular  vision,  but  invisible  in  monocular  vision,  i.e.  a  cyclopean 
bar. 

Methods 

The  equipment  is  diagrammed  in  Fig.  15.  The  procedure  was  as  just 
described  in  Section  3.7  except  that  the  rectangle  was  replaced  by  a  2.3  x  0.4  deg 
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vertical  bar  and  the  subject's  task  was  to  discriminate  whether  the  bar  was 
clockwise  or  anticlockwise  of  vertical. 

For  bars  whose  relative  disparity  placed  them  further  than  the 
surround  dots,  two  kinds  of  bar  were  generated.  This  can  be  imderstood  as 
follows.  For  such  a  bar  binocular  fusion  is  not  possible  beyond  a  critical 

disparity  5crit  given  by 

.  57h 

5crit=”5”  - 

where  h  is  the  width  of  the  aperture,  and  d  is  the  distance  from  the  eyes  to  the 
nearer  plane.  A  derivation  of  this  result  follows. 

Consider  the  situation  depicted  in  Fig.  18A  that  an  observer  uses  both  left 
(LE)  and  right  (RE)  eyes  to  view  an  indefinitely  extended  far  plane  (F)  through 
a  hole  of  width  h  in  an  indefinitely  extended  near  plane  (N).  In  Fig.  18A  the 
region  of  the  far  plane  visible  to  the  observer  has  an  uncrossed  disparity  of  5 
radians  with  respect  to  an  extended  surroimding  area  of  the  near  plane, 
where 

5=  a|  -  ^2 - (6) 

(angles  aj  and  a2  are  illustrated  in  Fig.  18B)  so  that 

^  “  d  (d+  s) 

In  Fig.  18A,  the  part  of  the  far  plane  marked  1  is  seen  by  the  left  eye 
oiUy,  and  the  part  marked  r  is  seen  by  the  right  eye  only.  The  width  of  the 
part  of  the  far  plane  that  is  seen  by  both  left  and  right  eyes  is  c,  where 

c  =  h  -  2e - (8) 
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Since 


e 


S  (i  -  h) 

2d 


(9) 


it  follows  from  equations  (8)  and  (9)  that 

s  (i  -  h) 


c  =  h 


(10) 


Hence 


SI 


d  +  s  d  d  (d  +  s) 
From  equations  (7)  and  (11) 

-5 _ 

d  +  s  d 


(11) 


(12) 


In  words:  equation  (12)  indicates  that  the  angle  subtended  by  the  part  of 
the  far  plane  that  is  seen  by  both  eyes  is  approximately  equal  to  the  angle 
subtended  by  the  hole  minus  the  relative  disparity  of  the  far  plane.  Thus, 
only  for  zero  disparity  is  there  no  part  of  the  far  plane  that  is  seen  by  one  eye 
only. 

Equation  (12)  indicates  that  there  is  a  critical  disparity  (Scjyj)  beyond 

which  there  is  no  part  of  the  far  plane  that  is  seen  by  both  eyes,  and  that 

s:  .  /I 


CRIT  J 


(13) 


In  degrees. 


57h 


CRIT 


(14) 
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The  narrower  the  hole  in  the  near  plane,  the  smaller  the  uncrossed 
disparity  beyond  which  the  target  cannot  be  fused.  Note  that  this  restriction  is 
geometrical  and  quite  distinct  from  the  physiological  limitations  usually 
discussed  under  the  heading  of  Panum's  fusional  area. 

The  situation  just  discussed  is  different  when  the  width  of  the  hole  in 
the  near  plane  in  Fig.  18A  is  equal  to  or  wider  than  the  intraocular  separation 
(i.e.,  h  >  i).  In  that  case,  a  finite  width  of  the  far  plane  is  seen  by  both  eyes, 
whatever  the  disparity  of  the  far  plane  relative  to  the  near  plane. 

Fig.  18C  illustrates  that  the  situation  described  above  for  imcrossed 
disparities  is  quite  different  for  targets  whose  disparities  are  crossed  in  that 
geometry  does  not  impose  a  limit  on  the  largest  crossed  disparity  that  can  be 
fused  for  a  target  of  a  given  width.  In  Fig.  18C  a  restricted  area  of  a  near  plane 
(N)  occludes  part  of  a  far  plane  (F).  The  entire  width  of  the  near  plane 
segment  is  seen  by  both  eyes  whatever  the  values  of  s,  d,  and  the  width  of  the 
near  plane.  The  parts  of  the  far  plane  that  are  seen  by  the  right  eye  only  or  by 
the  left  eye  only  are  marked  r  and  I  respectively.  The  parts  of  the  near  and  far 
planes  that  are  seen  by  both  eyes  in  the  geometrical  situation  illustrated  are 
marked  c^,  C2,  C3  and  C4. 

Results 

Data  points  in  Fig.  19A  plot  orientation  discrimination  thresholds  for  a 
constant-width  bar.  Bar  detection  threshold  is  indicated  by  the  vertical 
arrows).  The  vertical  dashed  lines  separate  the  disparity  range  over  which  the 
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bar  and  also  the  dots  surrounding  the  bar  were  seen  in  binocular  fusion  (BF) 
from  the  disparity  range  over  which  the  bar  was  diplopic  (BD).  Similar  data 
were  obtained  for  a  bar  that  obeyed  the  geometrical  constraints  expressed  in 
equation  5. 

In  the  final  experiment  we  carried  out  depth  matching  for  the 
Cyclopean  bar.  Results  are  shown  in  Fig.  20. 

Conclusions 

We  report  that  matched  depth  increased  smoothly  with  increasing 
uncrossed  disparity  through  a  range  over  which  orientation  discrimination 
threshold  fell  and  then  leveled  out.  In  some  instances  orientation 
discrimination  threshold  remained  approximately  constant  over  a  substantial 
range  of  disparities  while  depth  continued  to  increase  (note  that  the  abscissae 
are  logarithmic  in  Fig.  19).  We  conclude  that  the  processing  of  the  depth  of  a 
Cyclopean  form  is  dissociated  from  the  processing  of  the  orientation  of  that 
same  cyclopean  form. 

We  also  report  that  the  lowest  values  of  threshold  for  crossed  and 
imcrossed  disparities  are  similar  for  cyclopean  bars. 

Morgan  (1986)  found  that  cyclopean  vernier  acuity  is  in  the  region  of 
0.7  min  arc.  He  noted  that  this  is  roughly  10  times  finer  than  the  resolution 
limit  for  cyclopean  gratings  reported  by  Tyler  (1974),  and  pointed  out  that  this 
implies  —in  a  way  that  hyperacuities  for  monocularly-visible  form  cannot 
do —  that  the  precision  of  spatial  grain  in  the  cortex  is  less  than  1  min  arc 
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(Morgan,  1991).  Along  the  same  lines  we  reported  previously  that  aspect  ratio 
discrimination  for  cyclopean  rectangles  is  consistent  with  a  roughly  1.0  min 
arc  precision  of  encoding  the  height  or  width  of  a  cyclopean  rectangle,  again 
some  10  times  finer  than  grating  resolution  (see  above).  In  the  present  study, 
a  0.6  deg  change  in  the  orientation  of  a  bar  that  is  2.3  deg  long  corresponds  to  a 
1.4  min  arc  displacement  of  one  end  of  the  bar  relative  to  the  other  end,  thus 
providing  a  third  illustration  of  cyclopean  visual  performance  that 
substantially  transcends  the  limit  of  cyclopean  resolution. 

The  0.7  to  1.5  min  arc  acuities  for  DD  form  discussed  above  are  much 
inferior  to  the  2  to  5  sec  arc  acuities  for  LD  form,  just  as  the  roughly  3  c/deg 
grating  resolution  for  cyclopean  form  is  much  inferior  to  the  60  c/deg  grating 
resolution  for  luminance-defined  form.  However,  it  is  not  the  case  that  a 
given  spatial  discrimination  for  DD  form  is  necessarily  far  inferior  to  that 
same  discrimination  for  LD  form,  because  orientation  discrimination 
threshold  for  a  cyclopean  bar  falls  within  the  range  of  values  previously 
reported  for  luminance-defined  lines,  bars  or  gratings  (Andrews,  1967a,  b; 
Thomas  and  Gille,  1979;  Caelli  et  al.,  1983;  Ref.  127). 

3.9  Orientation  discrimination  for  texture-defined  form 

One  Paper  has  been  published  D.  Regan  "Orientation  discrimination 
for  texture-defined  bars"  Perception  (1995)  24. 

A  texture  pattern  consisting  of  short  (0.2  deg)  lines  contained  a  5.0  deg  x 
1.4  deg  texture-defined  bar.  The  bar  was  rendered  visible  by  the  orientation 
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difference  (20)  between  lines  inside  the  bar  and  outside  the  bar  (Fig.  21). 

Figure  22  shows  photographs  of  the  pattern.  A  luminance-defined  bar  was 
created  by  switching  off  all  lines  outside  the  bar.  Orientation  contrast  was  (p) 
equal  to  20  when  20  was  less  than  90  deg,  and  was  equal  to  (180  -  20)  when  20 
was  greater  than  90  deg. 

Orientation  discrimination  threshold  for  the  texture-defined  bar  was  a 
U-shaped  function  of  20  over  the  range  20  =  0  deg  to  20  =  180  deg  (Fig.  23). 

The  lowest  threshold  was  at  20  =  90  deg,  and  was  0.57  deg  for  both  observers 
tested.  Findings  were  similar  when  the  mean  orientation  of  the  texture  lines 
was  horizontal  rather  than  vertical.  This  orientation  discrimination 
threshold  was  a  little  different  for  the  lowest  threshold  for  motion-defined 
bars  (0.5  and  0.7  deg  for  two  observers.  Ref.  182)  or  for  cyclopean  bars  (0.6, 0.9 
and  0.9  deg  for  three  observers.  Ref.  238  and  Section  3.8  above)  or  for 
chromatic  gratings  (0.99  deg,  mean  of  two  observers,  Webster  et  al.,  1990), 
though  the  value  for  luminance  defined  form  in  the  present  study  (0.42  deg 
and  0.35  deg  in  Fig.  24)  were  a  little  lower  than  orientation  discrimination 
thresholds  for  texture-defined  bars  and  chromatic  gratings. 

Orientation  discrimination  threshold  for  the  luminance-defined  bar 
fell  to  a  limiting  value  as  the  luminance  contrast  of  the  bar  was  progressively 
increased  (Fig.  24,  dashed  lines).  Threshold  fell  considerably  more  rapidly  for 
the  luminance-defined  bar  than  for  the  texture-defined  bar  (Fig.  24, 
continuous  line),  though  the  asymptotic  values  were  not  greatly  different. 
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figure  21  A:  The  texture-defined  bar  had  an  orientation  (|>  that  could  be 
either  clockwise  (+)  or  anticlockwise  (-)  with  respect  to  the  vertical.  B,  C: 
Texture  lines  had  one  of  two  possible  orientation  (6)  that  were  symmetrical 
about  the  vertical. 
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Bgurg  2Z  Photographs  of  the  stimulus  display.  A:  Texture  lines  inside  and 
outside  the  bar  had  orientations  0  =  -10  deg  and  0  =  +10  deg,  respectively,  and 
the  bar  was  oriented  5  deg  anticlockwise  of  vertical  (i.e.,  <])  =  -5  deg).  B:  Texture 
lines  inside  and  outside  the  bar  had  orientations  of  0  =  -80  and  0  =  +80  deg 
respectively,  and  the  bar  was  oriented  5  deg  clockwise  of  vertical  (i.e.,  <(>  =  +5 
deg).  Twelve  new  texture  patterns  were  displayed  while  the  camera  shutter 
was  open.  The  clumped  black  areas  in  the  photograph  could  be  seen  as 
separated  individual  lines  when  the  monitor  was  viewed  by  eye. 
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ORIENTATION  CONTRAST  p  (deg) 

Figure  23  Orientation  discrimination  thresholds  (ordinate)  for  a  5.0  x  1.4 
deg  texture-defined  bar  are  plotted  as  ordinate  versus  the  orientation  contrast 
(P)  and  versus  the  orientation  (0)  of  the  0.2  x  0.03  deg  texture  lines.  Error  bars 
show  ±1  SE.  Where  no  error  bar  is  visible  the  error  bar  was  smaller  than  the 
data  point.  Vertical  arrows  indicate  bar  detection  threshold.  A  and  B  are  for 
observers  1  and  2  respectively. 
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BAR  VISIBILITY 

Figure  24  Orientation  discrimination  thresholds  for  a  5.0  x  1.4  deg  texture- 
defined  bar  (continuous  lines)  and  for  a  5.0  x  1.4  deg  luminance-defined  bar 
(dashed  lines)  are  plotted  along  the  ordinates.  Bar  visibility  (abscissa)  is 
quantified  as  a  multiple  of  bar  detection  threshold.  For  the  texture-defined 
bar  detection  threshold  is  expressed  as  orientation  contrast  (P),  and  for  the 
luminance-defined  bar  detection  threshold  is  expressed  as  luminance 
contrast.  A  and  B  are  for  observers  1  and  2  respectively. 
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The  orientation  discrimination  data  shown  in  figure  23  can  be 
discussed  in  terms  of  two  hierarchical  levels  of  processing,  one  operating  on  a 
considerably  larger  spatial  scale  than  the  other.  At  the  first  stage,  each  one  of  a 
parallel  array  of  orientation-tuned  neurons  with  small  classical  receptive 
fields  is  individually  excited  by  one  of  the  short  (0.2  deg)  lines  that  comprise 
the  texture.  At  the  second  stage  of  processing  the  outputs  of  this  parallel  array 
of  neurons  are  integrated  spatially.  This  stage  provides  a  basis  for  visually 
detecting  the  comparatively  large  (5  x  1.4  deg)  OTD  bar  and  for  discriminating 
its  orientation.  So  far  as  the  spatially-integrative  processing  of  bar  orientation 
is  concerned,  the  results  of  the  first  and  also  the  second  parts  of  experiment  1 
indicate  that  spatial  integration  is  not  greatly  different  whether  the 
integration  is  roughly  parallel  (e.g..  Fig.  22A)  or  roughly  perpendicular  (e.g.. 
Fig.  22B)  to  the  orientations  of  the  texture  elements. 

One  possible  explanation  for  the  finding  that  the  lowest  values  of 
orientation  discrimination  threshold  are  the  same  or  not  greatly  different  for 
OTD,  LD,  CD,  MD  and  DD  form  is  that  the  same  neural  mechanism 
determines  orientation  discrimination  threshold  for  spatial  form 
independently  of  which  of  the  five  kinds  of  contrast  is  responsible  for  the 
visibility  of  the  form.  An  alternative  hypothesis  is  that  orientation 
discrimination  threshold  for  the  five  kinds  of  form  is  determined  by  five 
different  neural  mechanisms,  and  that  the  similarity  between  the  thresholds 
is  a  result  of  early  visual  development  driven  by  the  infant's  persistent 
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attempts  to  achieve  eye-limb  coordination.  Since  an  object  is  an  object 
independently  of  how  it  is  detected  by  the  eye,  the  precision  of  orientation 
discrimination  required  of  the  organism  would  be  the  same  for  OTD,  LD,  CD, 
MD  and  DD  form.  These  two  hypotheses  are  discussed  elsewhere  (Refs.  155, 
201, 211). 

3.10  Texture-defined  letter  test 

One  paper  has  been  published.  D.  Regan  &  X.H.  Hong  "Recognition 
and  detection  of  texture-defined  letters"  Vision  Research.  34, 2403-2407  (1994). 

The  broad  issue  is  the  ability  to  see  and  recognise  an  object  such  as  a 
terrain  feature.  The  experiment  addresses  a  subject's  ability  to  recognise  an 
object  whose  visibility  is  created  by  texture  contrast  exclusively. 

Methods 

Stimulus  textiu'es  were  generated  by  an  IBM  486PC  clone  with 
ATI  VGA  "Wonder  plus  XL24"  graphics  card,  and  displayed  on  a  video 
monitor  (Sony  model  CPD  1304).  Resolution  was  640  (horizontal)  x  480 
pixels.  The  screen  measured  24  (horizontal)  x  18  cm  and,  at  the  viewing 
distance  of  2m,  subtended  6.8  x  5.1  deg.  so  that  one  pixel  subtended  0.64  min 
arc. 

The  screen  was  divided  into  80  (horizontal)  x  60  cells,  each  of  8  x  8 
pixels.  Any  given  cell  contained  one  dotted  bar.  Each  bar  consisted  of  four 
dots  as  illustrated  in  Fig.  25A  &  B.  (We  used  dotted  lines  to  avoid  the 
instrumental  artifact  that  the  luminance  of  a  continuous  line  depended  on  its 
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orientation).  In  Fig.  25A  &  B  the  black  squares  represent  illuminated  pixels, 
and  the  crosses  indicate  other  possible  locations  of  illuminated  pixels.  Pixels 
that  are  not  shown  black  or  crossed  were  never  illuminated.  Fig.  25A  shows 
one  of  the  four  possible  positions  of  a  vertical  bar,  and  Fig.  25B  shows  one  of 
the  four  possible  positions  of  a  horizontal  bar.  The  texture  could  be  degraded 
by  noise  by  adding  one  dot  to  every  cell,  two  dots  to  every  cell,  and  so  on  up  to 
11  dots  per  cell.  The  placement  of  the  noise  dots  within  a  cell  was 
randomized  from  cell  to  cell.  (A  noise  dot  placed  on  top  of  a  pre-existing  dot 
had  no  effect).  Thus,  in  every  8x8  pixel  cell,  there  were  16  possible  locations 
for  a  bar  dot  or  noise  dot  and  48  forbidden  locations. 

A  TD  letter  was  created  by  placing  one  horizontal  bar  within  every  cell 
in  the  letter  and  one  vertical  bar  within  every  cell  outside  the  letter.  With  all 
bars  horizontal  or  all  bars  vertical,  the  letter  was  perfectly  camouflaged.  Any 
vertical  or  horizontal  stroke  of  the  letter  had  a  width  of  5  cells.  The  letter  was 
drawn  within  a  square  of  28  x  28  cells,  so  that  at  the  viewing  distance  of  2m  a 
letter  subtended  2.4  x  2.4  deg.  A  reference  texture  was  defined  by  having 
either  one  vertical  bar  or  one  horizontal  bar  in  every  cell.  Fig.  26A  is  a 
photograph  of  the  stimulus  display  for  the  letter  H  with  zero  noise  dots  per 
cell.  In  Fig.  26B  &  C  the  number  of  noise  dots  per  cell  were  2  and  4 
respectively. 
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Egyrg  25  Each  one  of  the  4800  cells  comprised  an  array  of  8  x  8  pixels  .  Only 
the  pixels  marked  by  a  cross  or  a  black  square  could  be  illuminated.  Each  cell 
contained  a  vertical  bar  in  one  of  four  possible  positions  [e.g.  the  black 
squares  in  (A)]  or  a  horizontal  bar  in  one  of  four  possible  positions  [e.g.  the 
black  squares  in  (B)]. 
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Figure  26  Photographs  of  a  texture-defined  letter  (H)  whose  visibility  is 
progressively  degraded  by  adding  0, 2  &  4  noise  dots  per  cell  in  panels  A-D 
respectively. 
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Figure  27  Percent  correct  detection  (A-C)  and  recognition  (D-F)  of  texture- 
defined  letters.  The  horizontal  dashed  lines  indicate  chance  level  of 
performance.  A,  D,  observer  1.  B,  E,  observer  2.  C,  F,  observer  3. 
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Each  trial  comprised  two  one-sec  presentations,  only  one  of  which 
contained  a  letter.  During  the  other  presentation  of  a  trial  pair  every  8x8 
pixel  cell  contained  either  a  vertical  bar  or  a  horizontal  bar.  There  were  10 
letters:  H,  C,  D,  K,  N,  O,  R,  S,  V  and  Z.  The  number  of  noise  dots  per  cell  was 
identical  in  the  two  presentations.  The  monitor  screen  was  blank  except 
during  presentations.  The  two  presentations  were  separated  by  an  interval  of 
0.5  sec.  The  order  of  the  two  presentations  was  randomised. 

Subjects  were  instructed  to  state  whether  the  letter  was  presented  in  the 
first  or  in  the  second  trial  (letter  detection),  and  to  name  the  letter  (letter 
recognition). 

Results 

Detection  of  TD  letters 

For  all  subjects,  percent  correct  detections  remained  at  near-100%  from 
zero  up  to  5  or  6  noise  dots  per  cell,  after  which  detection  errors  increased 
progressively,  reaching  chance  (50%  correct)  at  9  or  10  noise  dots  per  cell. 

After  preliminary  practice  runs  that  established  this  pattern  of  scores,  letters 
with  from  0  to  3  noise  dots  per  cell  were  omitted  so  as  to  focus  on  the  portion 
of  the  psychometric  function  between  4  and  10  noise  dots  per  cell.  Detection 
scores  averaged  over  all  10  letters  for  each  of  the  three  subjects  are  plotted  in 
Fig.  27A-C.  Dotted  lines  are  for  the  case  that  the  same  texture  pattern 
remained  on  the  screen  throughout  any  given  presentation.  Continuous 
lines  are  for  dynamic  texture. 
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Recognition  of  TD  letters 

Letter  recognition  scores  averaged  over  all  10  letters  for  each  of  the 
three  subjects  are  plotted  in  Fig.  27D-F.  Dashed  and  solid  lines  have  the  same 
meaning  as  in  Fig.  27A-D.  Beyond  4  to  5  noise  dots  per  cell,  letter  recognition 
scores  fell  off  progressively,  reaching  chance  (10%  correct)  at  8  to  10  noise  dots 
per  cell. 

For  all  subjects,  the  curve  for  the  dynamic  texture  was  shifted  to  the  right  of 
the  curve  for  the  static  texture. 

One  possible  explanation  for  the  difference  between  results  for  static 
and  dynamic  textures  is  as  follows.  The  visibility  of  the  letters  was  degraded 
by  two  sources  of  noise:  variability  of  the  texture  across  the  pattern 
(Rubenstein  &  Sagi,  1990),  and  the  dots  that  we  added  deliberately.  In  the 
dynamic  pattern  case,  both  of  these  noise  components  were,  in  general, 
different  in  every  one  of  the  70  frames  in  one  presentation.  In  contrast,  every 
one  of  these  70  frames  contained  the  same  letter,  and  the  relation  between 
texture  inside  and  outside  the  letter  was  the  same.  Thus,  by  analogy  with  the 
process  of  time  domain  averaging  (reviewed  in  Regan,  1989,  pp.  47-50), 
summation  over  multiple  frames  due  to  temporal  integration  within  the 
visual  pathway  might  have  enhanced  signal-to-noise  ratio  and  hence 
improved  subjects'  scores. 

3.11  Modelling  the  detection  and  recognition  of  texture-defined  form 
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Results  have  been  reported  to  the  IEEE  and  a  complete  report  has  been 
published.  D.  Regan,  S.J.  Hamstra,  X.H.  Hong  &  S.  Kaushal  "Nonlinearities 
in  the  visual  processing  of  motion  and  form".  Proc.  JFEE  Conf.  on 
Mfln  &  Qybcrhtw?,  pp.  156-159,  (1993).  D.  Regan  &  X.H.  Hong  (1994).  "Two 
models  of  the  recognition  and  detection  of  texture- defined  letters  compared. 
Biological  Cybernetics.  72,  389-396. 

Outline  of  Two  Models 

Both  models  comprise  a  sequence  of  physiologically-plausible  filtering 
stages.  The  first  three  stages  and  the  final  stage  are  the  same  in  both  models. 
Model  1  has  only  one  nonlinear  stage,  and  this  distinguishes  it  from  model  2, 
which  has  a  second  nonlinear  stage.  The  effect  of  the  second  nonlinear  stage 
is  to  enhance  the  texture  boundary  by  comparing  texture  at  different  locations 
on  the  texture  pattern.  The  rationale  is  based  on  Nothdurft's  (1991, 1992) 
demoiTstration  that  segregation  between  two  orientation  textures  and  also  the 
perception  of  the  texture  boundary  arise  from  local  differences  in  orientation 
near  thg  border  rather  than  by  global  differences  in  the  orientation  statistics  of 
the  two  textures. 

Model  1,  illustrated  in  Figure  28 A,  comprises  the  following  sequence  of 
processing  stages:  (1)  parallel  orientation-selective  linear  spatial  filters;  (2)  full 
wave  rectification;  (3)  low-pass  spatial  filtering  (i.e.  blurring);  (4)  subtraction 
of  signals  from  filters  tuned  to  different  orientations;  (5)  either  a  letter 
detection  or  a  letter  recognition  stage.  Model  2,  illustrated  in  Fig.  28B  differs 
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diagrams  of  models  1  and  2  respectively.  The 
sequence  of  processing  depicted  is  applied  simultaneously  to  every  pixel  in 
the  stimulus  display. 
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Figure  29  Photograph  of  segregating  textures  with  a  single  boundary. 
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Figure  30  A:  Result  of  convolving  a  vertically-oriented  stage  1  DOG  filter 
with  the  Fig.  3A  pattern.  B:  As  for  A  except  that  the  filter  was  oriented 
horizontally.  C  and  D:  Rectified  versions  of  A  and  B  respectively.  E  and  F: 
low-pass  filtered  versions  of  C  and  D. 
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Figure  31  Filled  circles  plot  percent  correct  reading  accuracy  for  letter  H 
averaged  over  all  four  observers  (ordinate)  versus  the  number  of  noise  dots 
per  cell.  The  curves  in  A  and  C  plot  the  theoretical  predictions  of  model  1  (A) 
and  model  2  (C)  obtained  from  cross  correlations  weighted  by  cumulative 
normal  distributions.  Continuous  and  dashed  lines  were  obtained  using 
different  weighting  functions.  The  continuous  lines  in  B  and  D  plot 
predictiorrs  obtained  as  in  A  and  C  except  that  the  cross  correlations  were  not 
weighted. 
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Figure  32  The  first  three  stages  of  model  2  give  the  outputs  shown  in  Fig. 
4E  and  F.  The  output  of  the  stage  4  inhibition  is  shown  in  A  and  B.  Trace  C 
the  sum  of  traces  A  and  B.  See  text  for  details. 
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Figure  33  Filled  circles  plot  percent  correct  letter  detections  averaged  over 
all  ten  letters  and  all  four  observers  (ordinate)  versus  the  number  of  noise 
dots  per  cell.  The  continuous  lines  in  A  and  C  plot  the  theoretical  predictions 
for  models  1  and  2  respectively  using  the  weighted  signal-to-noise  ratio  of  a 
letter-segment  detector  to  model  the  letter  detection  stage.  The  continuous 
lines  in  B  and  D  plot  theoretical  predictions  based  on  the  unweighted  signal- 
to-noise  ratios. 
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from  model  1  in  that  nonlinear  iirhibition  follows  the  stage  3  low-pass 
filtering. 

Letter  Recognition  Predictions  and  Data 

First  we  describe  stage  1.  The  spatial  filtering  properties  of  visual 
pathway  neurons  have  been  modelled  by  several  functions  including  Gabor 
functions  and  difference  of  Gaussians  (Marceltja,  1980;  Daugman,  1980; 
Rodieck  &  Stone,  1965;  Parker  &  Hawken,  1988).  We  have  chosen  to  use 
difference  of  Gaussiairs  (DCX2)  filters  in  stage  1  on  the  basis  of  the  good  fit  with 
physiological  data  and  ease  of  computation.  We  believe  that  this  is  not  a 
critical  choice. 

The  DOG  filter  filled  an  11  x  11  pixel  square,  and  had  a  sensitivity  curve 
such  that  the  areas  above  and  below  zero  were  equal  so  that  the  filter  would 
not  respond  to  uiuform  illumination.  The  equation  of  the  DOG  filter  was 

y  =  3  exp  (-  (x/a)2)  -  2  exp  (-0.44(x/a)^) - (15) 

where  x  was  distance  expressed  in  pixels  and  <t  was  equal  to  1.5  pixels.  On  the 
grounds  that,  experimentally,  psychophysical  performance  on  letter  detection 
and  also  letter  recognition  were  considerably  affected  when  high  spatial 
frequencies  in  the  stimulus  were  attenuated  by  optical  blur,  the  value  of  a  was 
chosen  so  that  a  single  line  in  the  texture  pattern  would  strongly  stimulate 
the  DOG  filter  when  centred  on  the  filter.  (This  was  not  a  critical  choice 
however:  our  predictions  were  similar  when  a  was  1.0  and  3.0  pixels).  First,  a 
vertically-oriented  filter  was  convolved  with  the  stimulus  illustrated  in  Fig. 
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29,  giving  the  trace  shown  in  Fig.  30A  where  the  vertical  dashed  line  indicates 
the  vertical  texture  boundary.  Then  a  horizontally-oriented  filter  was 
convolved  with  the  stimulus  illustrated  in  Fig.  29,  giving  the  trace  shown  in 
Fig.  30B  where  the  vertical  dashed  line  indicates  the  same  texture  boundary. 

Stage  2  was  the  first  nonlinear  stage.  It  follows  Bergen  and  Adelson 
(1988),  and  is  equivalent  to  summing  the  outputs  of  two  half-wave  rectifiers, 
one  belonging  to  the  ON  system  and  one  to  the  OFF  system.  (Rectification 
models  the  fact  that  a  negative  firing  rate  is  impossible).  The  rectified  outputs 
of  stage  1  are  shown  in  Fig.  30C  and  D  respectively. 

At  stage  3  of  model  1,  the  Fig.  30C  and  D  traces  were  smoothed  by  a 
circularly-symmetric  Gaussian  filter  (Fig.  30E  and  F).  The  standard  deviation 
(a)  of  the  gaussian  was  10.5  pixels,  and  it  was  truncated  when  the  ordinate  fell 
to  1.0%  of  its  maximum  value,  giving  a  total  filter  diameter  of  63  pixels. 

We  modelled  the  behavioural  recognition  of  a  stimulus  letter  in 
terms  of  cross-correlatioirs  with  neural  templates  of  the  10  possible  letters. 

The  procedure  can  be  imderstood  as  follows.  The  letter  H  template  was 
placed  at  the  upper  left  corner  of  a  central  304  x  304  pixel  square,  and  the 
correlation  coefficient  was  calculated.  Then  the  template  was  translated  one 
pixel  rightwards,  the  correlation  coefficient  was  calculated  again,  and  so  on 
imtil  the  rightmost  edge  of  the  template  hit  the  right  edge  of  the  304  x  304 
pixel  square.  This  entire  procedure  was  repeated  with  the  template  displaced 
downwards  by  9  pixels,  and  so  on,  giving  a  total  of  9  left-to-right  passes.  We 
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noted  the  largest  value  of  the  resulting  correlation  coefficients.  The  same 
procedure  was  repeated  with  each  one  of  the  remaining  9  templates.  To 
allow  for  the  variability  between  one  texture  pattern  and  another  we 
repeated  this  procedure  for  5  different  texture  patterns  and  averaged  the 

results.  From  the  raw  correlations  we  calculated  normalized  correlations  (b). 
For  example,  for  the  Hy  stimulus  (i.e.,  letter  H  with  7  noise  dots  per  cell),  b  = 

(correlation  between  the  output  of  stage  4  for  the  Hy  stimulus  and  the 

template  of  a  given  letter) /(correlation  between  the  output  of  stage  4  for  the 
Hy  stimulus  and  the  H  template). 

Suppose  that  the  b  values  for  every  template  except  the  H  template  is 
zero.  Clearly,  letter  recognition  will  be  100%  correct.  Now  suppose  that  the  b 
values  are  1.0  for  letters  H  and  S,  but  are  zero  for  the  remaining  8  letters. 
Letter  recognition  will  be  50%  correct.  Again,  if  the  b  values  are  1.00  for 
letters  H,  S  and  R,  but  are  zero  for  the  remaining  7  letters,  letter  recognition 
will  be  33%  correct.  If  the  visual  system  is  noise-free,  letter  recognition  will 
change  abruptly  from  33.3%  to  50%  correct  if  the  b  value  for  letter  R  falls 
from  1.00  to  0.99.  But  if  the  visual  system  is  not  noise-free,  letter  recognition 
will  change  smoothly  and  continuously  from  33.3%  to  50%  correct  as  the  b 
value  for  letter  R  falls  progressively  from  1.0  through  lower  values.  We 
allowed  for  the  effect  of  noise  by  weighting  the  values  of  b  in  such  a  way  that 
b  values  were  attenuated  by  a  larger  and  larger  factor  as  they  fell  below  1.0. 
Following  Green  and  Swets  (1966)  we  chose  as  a  weighting  function  a 
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cumulative  normal  distribution  on  the  assumption  that  the  subject's  task 
can  be  modelled  as  discriminating  signal-plus-noise  from  noise. 

We  converted  the  ten  weighted  b  values  to  a  predicted  percent  correct 
letter  recognition  score  (lOORw)  by  writing 

100  Rw  =  100(l/i:[bw]) - (16) 

where  bw  was  the  weighted  value  of  b.  This  procedure  was  used  to  predict 
the  letter  recognition  score  for  all  the  letter  H  stimuli  (i.e.  Hq,  Hi, 

H2 . Hio). 

The  continuous  line  in  Fig.  31A  plots  the  predicted  reading  scores  for 
letter  H  using  a  weighting  fimction  whose  50%  and  75%  points  were 
respectively  at  b=0.88  and  b=0.92.  The  fit  with  experimental  data  (filled 
circles)  is  poor.  The  dashed  line  in  Fig.  31A  plots  predictions  obtained  with  a 
different  weighting  function  whose  50%  and  75%  points  were  respectively  at 
b=0.84  and  b=0.89,  and  illustrates  the  point  that  the  fit  with  data  cannot  be 
made  close  by  an  appropriate  choice  of  weighting  function. 

To  indicate  the  effect  of  the  cumulative  normal  weighting  function, 
the  continuous  line  in  Fig.  31 B  plots  R  as  left  hand  ordinate  where 

R=l/2:[b] - (17) 

The  only  difference  between  the  continuous  lines  in  Fig.  31A  and  B  is 
that  no  weighting  function  was  used  in  Fig.  31B. 

Now  we  will  discuss  the  predictions  of  model  2.  Model  2  is  illustrated 
in  Fig.  28B.  The  stage  3  low-pass  filter  is  crucial  in  model  2,  because,  as 


UNIVERSITE 

©  YORK 

UNIVERSITY 


96 


Department  of  Psychology 
FAX:  (416)  736-5814 


UNIVERSITE 

UNIVERSITY 
FACULTY  OF  ARTS 

4700  KEELE  STREET  .  NORTH  YORK  .  ONTARIO  •  CANADA  .  M3J  1P3 

illustrated  in  Fig.  32A  and  B,  it  allows  the  cross-orientation  inhibition  to 
operate  between  different  locations  on  the  texture  pattern  by  removing  energy 
from  the  left  of  the  texture  boimdary  in  Fig.  30E  and  from  the  right  of  the 
texture  boundary  in  Fig.  30F.  When  the  low-pass  filter  was  omitted,  the 
model's  predictions  fitted  the  psychophysical  data  very  poorly.  Furthermore, 
filters  with  a  standard  deviation  of  3.5  pixels  (diameter  23  pixels)  or  13.5  pixels 
(diameter  81  pixels)  gave  a  poorer  fit  with  data  than  did  the  filter  with  10.5 
pixels  standard  deviation  reported  here.  When  the  traces  in  Fig.  32A  and  B 
were  summed  (Fig.  32C),  the  texture  boundary  (vertical  dashed  line)  was 
clearly  marked  by  a  sharp  dip  in  the  curve.  In  contrast,  when  the  traces  in  Fig. 
32E  and  F  were  summed  (not  shown),  the  texture  boundary  was  not  evident. 

The  rationale  for  choosing  cross-orientation  inhibition  as  a  boundary¬ 
enhancing  process  is  based  on  two  physiological  lines  of  evidence.  First,  in 
htunan  subjects,  stimulation  by  superimposed  orthogonal  gratings  produces 
evoked  potentials  that  show  strong  nonlinear  inhibition(Morrone  &  Burr, 
1986;  Ref.  170).  Second,  cross-orientation  inhibition  is  well  established  at 
single-cell  level  in  visual  cortical  neurons  of  cat  and  monkey  and  cortical  cells 
can  show  orientation-selective  effects  from  stimuli  outside  the  classic  visual 
receptive  field  (Nelson  &  Frost,  1978;  Biu-r  et  al.,  1981;  Toyama  et  al.,  1981; 
Morrone  et  al.,  1982). 
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At  stage  4,  for  any  given  pixel,  the  output  of  stage  3  (ordinate  in  Fig. 
30E)  was  transformed  into  the  output  of  stage  4  (ordinate  in  Fig.  32A)  by 
means  of  equation  (18). 

y/y  =  yv  exp  (-kyh) - (18) 

where  yy  was  the  output  of  stage  3  and  y^  was  the  output  of  stage  4. 
Similarly,  equation  (23) 

y^h = yh®^p  (-^yv) - (19) 

was  used  to  transform  the  output  of  stage  3  (ordinate  in  Fig.  30F)  into  the 
output  of  stage  4  (ordinate  in  Fig.  32B),  where  yj^  was  the  output  of  stage  3 

and  y-^  was  the  output  of  stage  4. 

Letter  Detection  Predictions  and  Data 

We  assumed  that  the  subject's  detection  strategy  was  to  search  for  any 
part  of  a  letter  rather  than  to  search  for  the  entire  letter.  It  was  possible  for 
subjects  to  adopt  this  strategy  because  they  were  told  that  the  form  to  be 
detected  was  a  letter  and  were  shown  high-visibility  letters  before  they 
attempted  to  detect  low-visibility  letters.  This  prior  knowledge  allowed  them 
to  search  for  part  of  a  letter  because  they  already  knew  what  a  part  of  a  letter 
looked  like.  In  particular,  the  letters  were  composed  of  several  elongated 
segments.  We  modelled  this  "to  detect  a  part  is  to  detect  the  whole"  cognitive 
strategy  as  follows. 

The  letter-segment  detector  operated  as  follows.  We  took  a  bar-shaped 
region  whose  height  was  equal  to  the  height  of  a  letter  and  whose  width  was 
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equal  to  that  of  a  letter  stroke.  Texture  inside  and  outside  the  bar  consisted, 
respectively,  of  horizontal  and  vertical  lines.  At  the  input  to  the  letter- 
segment  detector,  a  texture-defined  bar  with  zero  noise  dots  per  cell  was 
represented  by  a  black  bar  of  width  48  pixels  surrounded  by  a  white  area  of 
almost  uniform  luminance.  As  the  number  of  noise  dots  was  progressively 
increased,  the  luminance  of  the  bar  and  its  surroundings  became 
progressively  more  patchy.  We  took  a  region  48  pixels  wide  whose  height 
was  equal  to  the  height  of  a  letter  and  placed  it  on  the  far  left  of  the  display  at 
the  input  to  the  letter-segment  detector.  The  value  of  DL  was  calculated.  We 
defined  DL  as  the  difference  between  the  highest  and  the  lowest  ordinate 
within  the  48  pixel  interval.  Then  we  moved  the  interval  one  pixel  to  the 
right  and  calculated  a  second  value  for  DL,  and  so  on.  We  defined  the  signal- 
to-noise  ratio  of  the  letter-segment  detector  as  equal  to  DLs/s,  where  DLs  is 

the  value  of  DL  when  the  interval  was  centred  on  the  texture-defined  strip, 
and  s  was  the  standard  deviation  of  all  the  values  of  DL  across  the  display. 

First  we  will  discuss  the  predictions  of  model  1.  The  continuous  line 
in  Fig.  33A  plots  the  detection  scores  predicted  by  weighting  the  signal-to- 
noise  ratios  with  a  cumulative  normal  distribution.  Filled  circles  plot 
experimentally-measured  detection  scores  averaged  over  both  all  10  letters 
and  all  4  subjects.  Fig.  33 A  shows  that  the  fit  between  prediction  and  data  was 
close. 
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The  continuous  line  in  Fig.  33B  plots  unweighted  signal-to-noise  ratio 
versus  the  number  of  noise  dots  per  cell.  To  compare  these  unweighted 
ratios  with  experimentally-measured  letter  detection  scores  we  set  the  chance 
detection  score  (50%)  on  the  right  hand  ordinate  equal  to  a  signal-to-noise 
ratio  of  zero  on  the  left  hand  ordinate.  We  then  set  the  signal-to-noise  ratio 
for  the  stimulus  with  6  noise  dots  per  cell  equal  to  the  experimentally- 
measured  detection  score  (98%  correct)  for  that  stimulus.  A  comparison  of 
Fig.  33 A  and  B  brings  out  the  effect  of  the  weighting.  The  shape  of  the 
weighting  function  was  almost  irrelevant.  Its  only  purpose  was  to  create  a 
saturation  or  ceiling  effect  so  as  to  render  all  signal-to-noise  ratios  over  about 
12  equivalent  to  a  100%  correct  score. 

The  corresponding  plots  for  model  2  (Fig.  33C  and  D)  were  similar.  Just 
as  for  model  1,  the  shape  of  the  weighting  function  was  almost  irrelevant;  its 
chief  function  was  to  provide  a  saturation  or  ceiling  effect. 

Discussion 

It  is  not  difficult  to  predict  letter  detection  data  if  we  assume  that  the 
subject's  detection  strategy  was  to  search  for  a  part  of  the  entire  letter.  Figure 
33  indicates  that  the  predictions  of  both  models  are  a  good  fit  with  the  data. 
Even  the  unweighted  signal-to-noise  ratios  fit  the  data  reasonably  well  for  6 
noise  dots  per  cell  and  upwards.  We  conclude  that  a  model  with  no  second 
nonlinearity  (model  1)  predicts  letter  detection  data  with  as  good  accuracy  as 
the  model  with  a  second  nonlinearity. 
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It  is  more  difficult  to  predict  letter  recognition  than  letter  detection 
data.  Model  1,  lacking  a  second  nonlinearity,  fails  badly  whatever  cumulative 
normal  weighting  function  is  chosen  (Figure  31A).  Recollect  that  the  effect  of 
the  second  nonlinear  stage  in  model  2  is  to  enhance  the  boundary  between 
different  textures  by  comparing  textures  at  different  locations  in  the  texture 
pattern.  For  model  2  the  fit  with  data  is  good  (Fig.  31C)  even  when  the  cross¬ 
correlations  are  not  weighted  (Fig.  31 D).  In  view  of  the  finding  that  some 
patients  with  multiple  sclerosis  (a  demyelinating  disease)  show  a  selective 
loss  of  ability  to  detect  and  read  texture-defined  letters  (see  Section,  3.13  below) 
it  may  be  significant  that  orientation-specific  lateral  interactions  in  monkey 
striate  cortex  are  transmitted  via  myelinated  horizontal  connections  (Gilbert 
et  al.,  1990). 

3.12  Normally-sighted  individuals:  Intersubject  variations  in  letter  reading 
tests  using  texture-defined  letters,  motion-defined  letters,  low  contrast 
luminance-defined  letters  and  high  contrast  luminance-defined  letters. 

A  paper  has  been  published  entitled  "Test-retest  variability  and  correlations 
between  tests  of  texture  processing,  motion  processing,  visual  acuity  and 
contrast  sensitivity"  by  T.  L.  Simpson  &  D.  Regan  in  Optometry  &  Vision 
Science,  (1995),  72, 11-16. 

The  broad  issue  is  the  ability  to  recognise  a  semi-camouflaged  terrain 


feature  by  means  of  texture  contrast. 
Methods 
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In  20  normally-sighted  subjects,  monocular  visual  acuity,  Regan  11% 
and  96%  letter  acuity,  Pelli-Robson  contrast  sensitivity,  and  thresholds  for 
recognising  motion-defined  letters  (MD  letters)  and  texture-defined  letters 
(TD  letters)  were  each  measured  twice  on  different  days. 

Results 

Test-retest  correlations  were  0.75,  0.91,  0.61,  0.90  and  0.84  for  high  and 
low  contrast  acuity,  contrast  sensitivity,  and  motion  and  texture  processing 
respectively. 

Conclusions 

Test-retest  variability  for  the  motion-defined  and  texture-defined  letter 
tests  were  at  least  as  low  as  for  established  clinical  tests  of  high-  and  low- 
contrast  acuity  and  contrast  sensitivity.  We  conclude  that  the  MD  and  TD 
letter  tests  offer  behavioural  and  clinical  information  that  not  only 
complements,  but  is  at  least  as  reliable  as,  information  provided  by 
conventional  visual  acuity  tests. 

3.13  Selective  loss  in  the  ability  of  patients  with  multiple  sclerosis  to  read 
texture-defined  letters,  motion-defined  letters  and  luminance-defined  lettprs 
Results  have  been  reported  to  ARVO  in  1994  and  a  paper  has  been  published. 
Simpson  TL  &  D  Regan.  "Specific  defects  in  the  visual  processing  of  texture 
in  patients  with  multiple  sclerosis".  Presented  at  the  Annual  meeting  of  the 
Association  for  Research  in  Vision  and  Ophthalmology  (ARVO),  Sarasota. 
Invest.  Ophthalmol.  &  Vis.  Sci.  35,  Suppl.  4,  p.  1257  (1994).  Regan  D& 
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Simpson  TL  "Multiple  sclerosis  can  cause  visual  processing  deficits  specific  to 
texture- defined  form”.  Neurology.  45, 809-815  (1995). 

The  broad  issue  is  the  ability  to  see  and  recognise  a  terrain  feature  by 
means  of  texture  contrast.  We  performed  the  following  tests  in  25  patients 
with  multiple  sclerosis  (MS)  and  25  age-matched  control  subjects:  recognition 
of  texture-defined  (TD)  letters;  recognition  of  motion-defined  (MD)  letters; 
recognition  of  luminance-defined  (LD)  letters  of  96%  and  11%  contrasts.  Six 
patients  with  normal  visual  acuity  were  abnormal  on  recognising  TD  letters, 
of  whom  one  gave  normal  restdts  on  all  other  tests.  Eleven  patients  were 
abnormal  on  MD  letter  recognition,  of  whom  four  gave  normal  results  on  all 
other  tests.  Visual  acuity  for  letters  of  11%  contrast  were  abnormally  low  in 
seven  patients,  of  whom  two  gave  normal  results  on  all  other  tests.  We 
conclude  that  the  neural  mechanisms  underlying  recognition  of  TD,  MD  and 
low-contrast  LD  letters  in  subjects  with  normal  visual  acuity  are  sufficiently 
different  that  they  can  be  differentially  damaged  by  MS.  Therefore,  TD,  MD 
and  LD  letter  tests  provide  complementary  irvformation.  We  suggest  that  the 
detection  of  TD  letters  can  be  disrupted  by  demyelination  of  long-range 
horizontal  connections  between  orientation-timed  neurons  in  striate  cortex. 
3.14  Parallel  processing  of  three  visual  dimensions 

One  report  has  been  published  "Parallel  independent  encoding  of  orientation, 
spatial  frequency  and  contrast"  by  A.  Vincent  &  D.  Regan  Perception  (1995)  24, 
491-499. 
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These  two  experiments  address  the  cognitive  issue  of  attentional 
resources.  A  considerable  body  of  experimental  evidence  supports  the 
hypothesis  that  early  visual  processing  of  spatio-temporal  information  can  be 
modelled  in  terms  of  parallel  psychophysical  chaimels.  In  such 
psychophysical  models  of  early  visual  processing,  retinal  image  information 
typically  passes  through  an  array  of  filters,  each  of  which  is  tuned  to  temporal 
frequency,  spatial  frequency  and  orientation.  The  question  of  possible 
interactions  between  the  processing  of  temporal  and  spatial  information  has 
been  primarily  investigated  by  using  techniques  of  masking,  adaptation  or 
suprathreshold  summation,  all  of  which  rely  on  an  analysis  of  how  the 
presence  of  one  stimulus  affects  a  subject's  ability  to  detect  a  second  stimulus. 
The  weight  of  evidence  does  not  support  the  idea  that  the  orientation,  spatial 
frequency  and  temporal  tuning  of  these  early  visual  filters  is  separable  (i.e. 
independent). 

However,  the  majority  of  the  studies  cited  above  focused  on  the 
detection  of  gratings  of  different  spatial  modulation  frequencies,  spatial 
frequencies  and  orientations,  and  there  is  evidence  that  the  results  of 
experiments  on  discrimination  cannot  necessarily  be  predicted  from  the 
results  of  experiments  on  detection.  For  example,  in  spite  of  the  evidence  just 
cited  that  the  orientation  and  spatial  frequency  tuning  of  early  visual  filters 
are  not  independent,  we  have  shown  that  orientation  discrimination 
thresholds  for  pairs  of  gratings  differ  widely  in  spatial  frequency  are  as  low  as 
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orientation  discrimination  thresholds  for  pairs  of  gratings  of  the  same  spatial 
frequency,  and  spatial  frequency  discrimination  threshold  for  pairs  of 
orthogonal  gratings  are  as  low  as  for  pairs  of  parallel  gratings  (Ref.  127). 

A  limitation  of  earlier  discrimination  studies  is  that  they  did  not 
involve  the  temporal  dimension.  As  well,  most  of  these  studies  have 
required  the  subject  to  discriminate  the  orientation  of  two  considerably 
different  spatial  frequencies  or  to  discriminate  the  spatial  frequency  of  two 
considerably  different  orientations.  This  experimental  design  requires 
subjects  to  process  only  one  dimension  at  a  time,  and  to  filter  out  changes  in 
the  task —  irrelevant  dimension.  Such  experiments  fail  to  address  the  further 
question  whether  there  are  interactions  between  the  processing  of  orientation 
and  spatial  frequency  when  they  are  processed  simultaneously.  I  report  here 
two  empirical  studies.  One  was  on  the  extent  to  which  observers  can 
unconfound  simultaneous  changes  in  the  temporal  frequency,  spatial 
frequency  and  orientation  of  a  grating  when  they  are  required  to  encode  all 
three  visual  dimensions  simultaneously.  In  a  second  study  we  measured  the 
extent  to  which  observers  can  unconfound  simultaneous  changes  in  the 
contrast,  spatial  frequency  and  orientation  of  a  grating  when  they  are  required 
to  encode  all  three  visual  dimensions  simultaneously. 

The  method  of  constant  stimuli  was  used  in  both  studies.  Trial  stimuli 
were  organized  as  a  cube  of  6  x  6  x  6  cells.  Each  of  the  216  ceUs  represented  a 
different  grating.  The  three  visual  dimensions  varied  orthogonally  along  the 
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x-axis,  y-axis  and  z-axis  of  the  stimulus  cube.  The  216  test  stimuli  were 
presented  in  random  order,  one  per  trial.  After  each  trial  the  observers  were 
required  to  make  three  judgements.  For  each  of  the  visual  dimensions  in 
turn  the  observer  judged  whether  the  test  value  was  greater  or  smaller  than 
the  mean  of  the  set  of  216  stimuli.  In  subsidiary  experiments  only  one  visual 
dimension  at  a  time  was  varied  so  that  observers  made  only  one  rather  than 
three  discriminations  after  each  trial. 

The  task  required  of  our  subjects  involved  memory  and  selective 
attention  as  well  as  visual  perception.  Variations  of  the  three  dimensions 
were  totally  confounded  within  the  set  of  216  stimuli.  The  subject's  task  was 
to  unconfound  and  to  discriminate  the  simultaneous  variations  of  all  three 
variables  with  respect  to  reference  values  stored  in  memory.  Following  a 
previous  line  of  argument  (Ref.  150)  we  can  model  this  task  as  follows:  (1) 
build  up  a  memory  of  the  mean  values  of  all  three  variables  for  the  set  of  216 
gratings;  (2)  neurally  encode  and  store  in  memory  the  values  of  the  three 
variables  presented  in  any  given  test  grating;  (3)  selectively  attend  to  the 
newly-stored  representation  of  variable  1,  and  compare  it  with  the  stored 
representation  of  the  mean  value  of  variable  1  of  the  set  of  216  gratings;  (4) 
selectively  attend  to  the  newly-stored  representation  of  variable  2,  and 
compare  it  with  the  stored  representation  of  the  mean  value  of  variable  2  for 
the  set  of  216  gratings;  (5)  selectively  attend  to  the  newly-stored  representation 
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of  variable  3  and  compare  it  with  the  stored  representation  of  the  mean  value 
of  variable  3  for  the  set  of  216  gratings. 

Using  multiple  stepwise  regression  analysis  we  found  that 
observers  could  almost  perfectly  unconfound  simultaneous  variations 
in  three  variables. 

Figs.  34  and  35  show  the  results  of  a  different  kind  of  analysis  of 
the  same  data  in  which  we  obtained  nine  psychometric  functions 
simultaneously. 

However,  even  though  subjects  were  able  to  ignore  variations  in 
the  two  task-irrelevant  variables,  it  does  not  necessarily  follow  that 
discrimination  threshold  for  the  task-relevant  variable  was  imaffected 
by  the  simultaneous  changes  in  the  task-irrelevant  variables.  A 
plausible  possibility  is  that  orientation  discrimination  threshold  would 
be  lower  when  the  only  trial-to-trial  variations  were  in  the  task¬ 
relevant  variable  than  when  all  three  variables  changed 
simultaneously.  In  the  first  case  the  reference  value  of  only  one 
variable  would  need  to  be  stored  in  memory,  and  in  addition  subjects 
could  focus  their  entire  attentional  resources  on  the  task-relevant 
variable  alone,  while  in  the  second  case  the  reference  values  of  the 
three  variables  would  need  to  be  stored  in  memory,  and  in  addition 
subjects  must  distribute  their  attentional  resources  among  three 
variables.  If  we  consider  only  the  memory  aspect,  there  is  reason  to 
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Figure  34  Simultaneous  encoding  of  spatial  frequency,  orientation  and 
contrast.  Nine  psychometric  functions  were  obtained  simultaneously.  A,  B, 
C:  Orientation  discrimination  task.  The  percentage  of  "test  grating 
orientation  clockwise  with  respect  to  the  mean  of  the  stimulus  set"  responses 
are  plotted  as  ordinate  test  grating  orientation  (A),  spatial  frequency  (B)  and 
contrast  (C).  D,  E,  F;  Spatial  frequency  discrimination  task.  The  percentage  of 
"test  grating  spatial  frequency  higher  than  the  mean  of  the  stimulus  set"  is 
plotted  as  ordinate  test  grating  orientation  (D),  spatial  frequency  (E)  and 
contrast  (F).  G,  FI,  I:  Contrast  discrimination  task.  The  percentage  of  "test 
grating  contrast  higher  than  the  mean  of  the  stimulus  set"  is  plotted  as 
ordinate  ys  test  grating  orientation  (G),  spatial  frequency  (FF)  and  contrast  (I). 
The  mean  orientation  of  the  stimulus  set  was  40  deg  clockwise  of  vertical.  On 
the  spatial  frequency  axis,  1.0  corresponds  to  the  mean  spatial  frequency  of  5.0 
c/deg.  On  the  contrast  axis,  1.0  corresponds  to  the  mean  contrast  of  0.5. 
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Figure  35  Simultaneous  encoding  of  spatial  frequency,  orientation  and 
temporal  frequency.  The  observer  was  presented,  one  by  one,  with  216 
stimulus  gratings,  each  of  which  had  a  different  combination  of  spatial 
frequency,  orientation  and  temporal  frequency  and  instructed  to  signal  how 
spatial  frequency  (A-C),  orientation  (D-F)  and  temporal  frequency  (G-I) 
differed  from  the  mean  values  for  the  set  of  216  stimuli.  Each  of  the  three  sets 
of  responses  are  plotted  versus  normalized  spatial  frequency  (A,  D,  G), 
orientation  (B,  E,  H)  and  normalized  temporal  frequency  (C,  F,  I). 
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expect  some  interaction  of  spatial  and  temporal  components  of  the  memory 
trace  (Dutta  &  Nairne,  Memory  &  Cognition,  1993,  21,  440-448). 

Nevertheless,  however  plausible  it  might  seem,  this  possibility  was 
ruled  out  empirically.  Our  finding  that  thresholds  were  approximately  the 
same  whether  trial-to-trial  changes  occurred  in  the  values  of  only  one  or  all 
three  variables  leads  us  to  conclude  that  the  possible  limitations  of  memory 
and  attentional  resources  just  described  are  not  evident,  and  also  that 
interactions  between  the  processing  of  temporal  frequency,  spatial  frequency 
and  orientation  are,  at  most,  very  small  even  when  the  three  visual 
dimensions  are  processed  simultaneously — at  least  near  discrimination 
threshold. 

On  the  grounds  that  the  three  visual  dimensions  varied 
simultaneously  and  subjects  made  their  responses  after  the  test 
presentation,  we  suggest  that  the  temporal  frequency,  spatial  frequency 
and  orientation  of  any  given  grating  are  encoded  in  parallel  as  are 
contrast,  spatial  frequency  and  orientation. 

One  implication  of  our  conclusion  that  both  sets  of  variables  are 
encoded  independently  in  parallel  and  with  negligible  cross  talk  can  be 
understood  as  follows.  Cells  in  primary  visual  cortex  are  in  general  not 
selectively  sensitive  to  one  visual  dimension  only.  Rather,  they  respond  to 
temporal  frequency,  contrast,  spatial  frequency  and  orientation  —  though  the 
balance  of  sensitivities  varies  from  cell  to  cell.  As  discussed  previously  (Refs. 
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STORK 
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147, 155),  our  proposed  physiological  explanation  for  these  psychophysical 
findings  is  that  information  about  these  four  dimensions,  though 
confounded  at  single-cell  level  in  striate  cortex,  is  unconfounded  by  different 
connectivities  within  the  cell  population. 

Psychophysical  evidence  for  a  neural  mechanism  that  encodes  angles 
Results  have  been  reported  to  ARVO  (1995)  and  a  report  has  been  published 
"Evidence  for  a  neural  mechanism  that  encodes  angles"  by  D.  Regan,  R.  Gray 
&  S.  Hamstra  Vision  Research  (1996)  36, 323-330. 

It  is  well  known  that  the  orientation  of  a  line  is  a  variable  so  important 
that  it  merits  the  anatomical  substrate  of  orientation  columns,  a  substrate  so 
gross  that  it  can  be  rendered  visible  to  naked-eye  examination  of  the  visual 
cortex.  Here  we  report  a  search  for  evidence  that  there  is  a  neural  mechanism 
for  angle  that  is  distinct  from  the  mechanism  for  orientation. 

We  measured  the  discrimination  threshold  (A0)7h  for  angle  0,  where  0 
was  either  the  angle  of  a  Vee  composed  of  two  straight  lines  contained  within 
the  frontoparallel  plane  or  the  angle  of  intersection  of  two  straight  lines 
contained  within  the  frontoparallel  plane.  The  two-line  pattern  was  rotated 
bodily  through  a  random  angle  between  trials  with  the  aim  of  eliminating  the 
absolute  orientation  of  one  or  the  other  line  as  a  reliable  cue  to  the  task.  We 
demonstrated  that  this  aim  was  achieved  by  showing  that  orientation 
discrimination  threshold  for  a  single  line  was  very  high  in  our  situation  of 
jitter  rotation.  Our  main  conclusion  is  that  the  ability  to  discriminate  a 
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change  in  angle  0  cannot  entirely  be  explained  in  terms  of  the  ability  to 
discriminate  changes  in  the  orientations  of  the  individual  lines  that  comprise 
the  Vee.  We  propose  that  the  human  visual  pathway  contains  a  neural 
mechanism  that  encodes  the  difference  in  the  orientations  of  two 
simultaneously-presented  straight  lines.  Discrimination  threshold  for  angle 
(A0Th)  is  roughly  twice  orientation  discrimination  threshold  for  an  isolated 
line  when  mean  orientation  is  near-vertical.  When  subjects  cannot  use  the 
orientation  of  one  or  other  line  as  a  cue  to  the  task,  the  plot  of  (A0)Th  versus  0 
is  approximately  flat  between  0  =  20  deg  and  0  =  160  deg:  although 
discrimination  threshold  at  a  reference  angle  of  90  deg  is  significantly  lower 
than  at  an  arbitrary  reference  angle,  the  difference  is  small  (<30%). 

At  first  sight  our  failure  to  find  that  a  right  angle  is  special  seems  out  of 
line  with  one's  intuitions.  However,  our  finding  can  be  stated  as  "when 
feedback  is  provided  and  observers  are  given  time  to  learn  the  task, 
discrimination  threshold  is  only  a  little  lower  at  90  deg  than  at  other  angles". 
When  we  required  observers  to  set  angles  without  feedback  and  with  no 
preliminary  display  of  the  designated  angle,  performance  was  much  better  at 
90  deg  than  at  other  angles.  However,  angle-dependent  differences  were 
almost  abolished  when  there  was  a  preliminary  display  of  the  designated 
angle  or  when  feedback  was  provided. 

3.16  "Accuracy  of  reproducing  angles:  is  a  right  angle  special"  by  R.  Gray  & 
D.  Regan  Perception  (1996)  25, 531-542.. 
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We  compared  the  ability  to  set  the  angle  of  a  Vee  to  a  designated  value 
in  the  following  three  conditions:  (a)  verbal  designation  of  Vee  angle;  (b) 
initial  30-sec  visual  demonstration  of  the  designated  Vee  angle;  (c)  verbal 
designation  of  Vee  angle  plus  feedback  after  every  setting.  The  designated 
angles  were  90  deg  and  45  deg  plus  three  arbitrary  angles  (65  deg,  125  deg  and 
145  deg).  Each  run  comprised  thirty  consecutive  settings.  To  ensure  that  our 
observers  based  their  settings  entirely  on  Vee  angle  we  arranged  that  line 
orientation  did  not  provide  a  reliable  cue  to  Vee  angle.  In  condition  (a), 
accuracy  of  setting  Vee  angle  was  significantly  worse  when  the  designated 
angle  was  other  than  90  deg  or  45  deg.  This  was  not  the  case  in  condition  (b), 
indicating  that  observers  maintained  a  memory  of  the  initial  demonstration 
throughout  a  rim  of  30  settings.  Setting  error  was  not  significant  in  condition 
(c)  for  any  of  the  five  angles.  However,  even  in  condition  (c),  setting-to- 
setting  variability  was  significantly  lower  for  the  90  deg  angle  than  for  the 
other  angles. 

3.17  MEG  study  on  colour-defined  form 

This  experiment  was  completed  during  the  1993-96  funding  period,  but  most 
of  the  work  was  carried  out  during  the  1990-93  funding  period,  and  reported 
in  the  1991-93  Final  Report  for  grant  #  AFOSR-91-0080,  so  the  present  report 
will  be  restricted  to  noting  that  a  paper  has  been  published  entitled  "Magnetic 
and  electrical  responses  to  chromatic  contrast  in  human"  by  D.  Regan  &  P.  He 
Vision  Research  (1996)  36, 1-18. 
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3.18  MEG  study  on  texture-defined  form 

This  experiment  was  completed  during  the  1993-96  funding  period,  but  most 
of  the  work  was  carried  out  during  the  1990-93  funding  period  and  reported  in 
the  1990-93  Final  Report  for  grant  #  AFOSR-91-0080,  so  the  present  report  will 
be  restricted  to  noting  that  a  paper  has  been  published  entitled  "Magnetic  and 
electrical  responses  of  the  human  brain  to  texture-defined  form  and  to 
textons"  by  D  Regan  &  P  He  Journal  of  Neurophysiology  (1995)  74, 1167-1178. 

3.19  MEG  evidence  for  an  audio-visual  convergence  area  in  the  human 
brain 

A  report  has  been  published  entitled  "An  audio-visual  convergence  area  in 
human  brain"  by  M.  P.  Regan,  D.  Regan,  &  P.  He  .Experimental  Brain 
Research.  (1995)  106, 485-487. 

The  purpose  of  this  experiment  was  to  find  an  area  in  the  human  brain 
where  visual  and  auditory  information  are  integrated. 

Rationale 

Suppose  a  subject  views  a  light  flickering  at  frequency  Fy  Hz  while 
listening  to  a  sound  whose  amplitude  is  modulated  at  frequency  Fa  Hz. 
Neural  processing  within  the  visual  pathway  can  generate  responses  at  Fy, 

2Fy,  3Fy,  4Fy - Hz,  and  neural  processing  within  the  auditory  pathway  can 

generate  responses  at  Fa,  2Fa,  3Fa,  4Fa, - Hz  but  combination  frequencies 

cannot  be  generated  until  visual  and  auditory  signals  converge.  Therefore, 
combination  frequencies  are  a  "signature"  of  an  audio-visual  convergence 
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area  of  the  brain.  [Combination  frequencies  are  (nFi  ±  mF2),  where  n  and  m 
are  nonzero  integers)]. 

Methods  and  Results 

Subject  fixated  the  centre  of  a  uniform  circular  patch  of  light  whose 
luminance  was  sinusoidally  modulated  at  frequency  Fy  =10.304  Hz.  A  1000 
Hz  sinewave  was  multiplied  by  a  sinewave  of  frequency  Fa  =  5.054  Hz  and  the 
resulting  waveform  was  transduced  into  an  amplitude-modulated  (AM)  tone 
and  fed  into  the  subject's  ear. 

Magnetic  fields  produced  by  brain  cells  in  response  to  sensory 
stimulation  were  recorded  using  an  array  of  7  coils  placed  close  to  the  subject's 
head.  Because  the  amplitude  of  the  brain's  magnetic  field  is  exceedingly 
small,  the  recording  coils  were  immersed  in  liquid  helium  to  reduce 
thermally-generated  noise  in  the  coils,  and  both  the  subject  and  the  recording 
equipment  were  placed  in  a  magnetically-shielded  room.  During  a  recording 
session,  the  subject  remained  still,  while  the  array  of  recording  coils  was 
moved  to  different  locations  on  the  head  (typically,  49  coil  locations  in  all). 
Our  major  technical  problem  was  that  the  power  of  the  audio-visual  cross¬ 
modulation  response  was  some  thousands  of  times  less  than  the  total  power 
of  the  brain's  magnetic  field.  We  countered  this  problem  by  analysing  the 
brain's  magnetic  field  into  a  power  spectrum  at  the  theoretical  limit  of 
frequency  resolution  (Gabor,  1946)  using  the  "nondestructive  zoom-FFT" 
technique  whose  theoretical  basis  and  practical  implementation  we  have 
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described  elsewhere  (Refs.  167, 177, 178).  In  the  present  case,  each  recording 
had  a  duration  of  160  sec  and  the  spectrum  had  a  resolution  of  0.0078  Hz  over 
the  recording  bandwidth  of  1  to  50  Hz.  This  technique  cleanly  extracts  very 
small  signals  from  noise  by  capitalising  on  the  finding  that  the  magnetic  (and 
electrical)  responses  of  the  brain  to  repetitive  stimulation  are  sufficiently 
constant  in  both  amplitude  and  delay  that  the  bandwidth  of  a  harmonic  or 
cross-modulation  sensory  response  component  can  be  as  narrow  as  0.002  Hz 
or  less.  In  our  case  the  noise  was  distributed  across  the  6274  lines  of  the  1-50 
Hz  spectrum  while  the  audio-visual  response  was  concentrated  into  a  single 
0.0078  Hz  frequency  interval  in  a  relatively  noise-free  part  of  the  spectrum,  so 
that  the  signal-to-noise  enhancement  was  several  thousand  times.  Because 
this  technique  is  very  dependent  on  the  stability  of  the  stimulation 
modulation  frequencies,  the  modulation  frequencies  of  both  visual  and 
auditory  stimuli  were  generated  by  crystal-controlled  synthesizers. 

Stimulation  was  as  follows.  Subjects  fixated  the  centre  of  a  uniform 
circular  area  of  green  light  whose  luminance  was  modulated  sinusoidally  at 
Fy  =  10.304  Hz  to  a  depth  of  100%.  A  1000  Hz  sinewave  tone  was  amplitude- 
modulated  at  a  frequency  =  5.054  Hz,  and  fed  into  the  subject's  left  ear 
while  the  subject  viewed  the  flickering  light.  (The  choice  of  stimulus 
frequencies  was  not  crucial  within  broad  limits,  but  0.001  Hz  stability  was 
essential).  Fig.  43A  is  a  contour  map  over  the  skull  of  the  2Fv  Hz  component 
of  the  brain's  magnetic  field.  The  contour  lines  join  locations  over  the  skull 
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FjgUfg  36  Contour  maps  of  magnetic  response  amplitude  recorded  from 
the  human  brain  during  simultaneous  visual  and  auditory  stimulation.  A: 
responses  generated  in  visual  cortex.  B:  responses  generated  in  auditory 
cortex.  C:  responses  generated  in  an  audio-visual  convergence  area.  The 
circle  round  the  map  is  a  best  fit  to  fit  the  curvature  of  the  skull  over  the 
region  of  the  head  from  which  recordings  were  made,  as  illustrated  in  the 
heads  at  the  far  right.  Recording  sites  are  marked  +  and  -  according  to  the 
phase  of  the  response.  The  viewpoint  was  from  the  left  side  of  the  head  in  A 
&  B  and  from  directly  behind  the  head  in  C.  The  dots  and  arrows  in  the  small 
stylized  heads  mark  the  location  and  magnetic  field  direction  of  the  source  of 
the  brain  response  from  three  different  viewpoints.  The  inion  is  marked  in 
A.  The  left  periauricular  point  is  marked  in  A  &  B. 
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where  response  amplitude  was  constant.  As  expected  from  the  fact  that  the 
recording  coils  emphasize  the  contributions  of  shallow  nearby  sources,  the 
response  to  visual  stimulation  is  concentrated  near  the  primary  visual  cortex 
of  the  brain  over  the  occipital  region  at  the  back  of  the  head.  Fig.  36B  maps 
the  amplitude  of  the  2?^  component  of  the  magnetic  field.  The  field 
distribution  is  a  reasonable  fit  to  that  of  a  single  dipole  (r  =  0.92).  The  location 
of  the  dipole  is  illustrated  in  the  three  small  stylized  heads.  As  expected,  the 
estimated  location  of  this  auditory  response  source  is  close  to  the  location  of 
primary  auditory  cortex  in  human  (Galabarda  &  Sandies,  1980). 

The  coordinates  of  the  equivalent  dipoles  in  Fig.  36B  &  C  were  as 
follows  (X,  Y  &  Z  in  cm).  B:  X  =  1.83;  Y  =  4.73;  Z  =  6.82;  V  =  -154  deg.  C:  X  =  - 
0.22;  Y  =  4.31;  Z  =  6.56;  V  =  170  deg.  The  coordinate  system  has  been  described 
previously  (Williamson  &  Kaufman,  1989;  Williamson  et  al.,  1991).  In  brief,  a 
sensor  position  indicator  was  used  not  only  to  record  coil  locations  but  also  to 
encode  head  dimensions  and  shape.  The  multiple  recordings  were  translated 
into  a  map  of  the  magnetic  field  over  a  spherical  surface  computed  to  be  a  best 
fit  to  the  skull  from  which  the  recordings  were  made.  Source  location  was 
computed  in  terms  of  Cartesian  coordinates  defined  by  the  nasion  and 
periauricular  points.  The  origin  is  the  mid-point  between  left  and  right 
periauricular  points.  The  X-axis  passes  through  the  origin  to  emerge  from  the 
head  at  the  nasion.  The  Z-axis  is  oriented  perpendicular  to  both  the  X-axis 
and  the  line  between  the  periauricular  points,  and  passes  through  to  emerge 
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near  the  vertex.  The  Y-axis  is  perpendicular  to  both  the  X-  and  Z-axes,  and 
passes  through  the  origin  to  emerge  from  the  left  hemisphere.  The 
orientation  of  the  equivalent  dipole's  moment  is  given  by  angle  T'  measured 
counterclockwise  from  the  plane  containing  the  Z-axis  and  the  location  of  the 
dipole. 

Our  main  finding  is  shown  in  Fig.  36C  where  the  contours  join  scalp 
locations  at  which  the  amplitude  of  an  audio-visual  (2Fa  +  Fy)  Hz  cross¬ 
modulation  response  had  constant  amplitude.  These  cross-modulations 
responses  are  the  first  objective  evidence  for  an  audio-visual  convergence 
area  in  human  brain  cortex.  Since  the  recording  coils  emphasize  nearby 
sources,  we  conclude  that  this  convergence  area  is  far  from  visual  cortex  and 
close  to  the  source  of  the  auditory  response  (compare  the  viewpoints  of  the 
maps  in  Fig.  36A,  B  &  C).  The  entire  experiment  was  repeated  two  months 
later  and  gave  very  similar  results.  Similar  results  to  those  shown  in  Fig.  36 
were  obtained  from  a  second  subject. 

In  the  absence  of  previous  data  on  audio-visual  convergence  areas  in 
human  brain  we  can  compare  our  data  only  with  the  known  locations  of 
sensory  convergence  area  in  the  cortex  of  macaque  monkey.  The  field 
distribution  in  Fig.  36C  is  a  close  (r  =  0.92)  fit  to  that  of  a  single  dipole  whose 
location  is  depicted  graphically  in  the  three  stylized  heads  in  Fig.  36C.  Bearing 
in  mind  that  the  magnetic  field  that  we  recorded  is  largely  generated  within 
the  bodies  of  cortical  neurons  aligned  parallel  to  the  skull  (Hari  and 
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Lounasna,  1989;  Williamson  et  al.,  1991),  this  eqviivalent  dipole  location 
corresponds  to  a  known  cortical  audio-visual  convergence  area  in  macaque 
mordcey  (Tigges  and  Tigges,  1985). 

Studies  on  the  auditory  processing  of  complex  waveforms 

It  is  widely  accepted  that  a  considerable  proportion  of  the  information 
transmitted  in  speech  is  transmitted  via  changes  in  the  frequencies  and 
amplitude  of  formants  (formants  are  well-defined  peaks  or  concentrations  of 
power  in  the  frequency  spectrum).  Much  of  the  research  on  speech  processing 
and  comprehension  compares  psychophysical  or  cognitive  responses  with  the 
auditory  waveform  or  spectrum  that  reaches  the  ear.  We  propose  that  further 
insights  might  be  possible  by  comparing  psychophysical  responses,  not  with 
the  auditory  stimulus  that  reaches  the  ear,  but  rather  with  the  signal  that  is 
available  to  the  brain,  i.e.  the  output  of  the  hair  cells.  The  force  of  this  point 
is  that  the  hair  cell  transducer  function  resembles  a  saturating  rectifier  and, 
therefore,  distorts  and  changes  the  incoming  waveform  very  strongly,  and  in 
a  way  that  is  far  from  intuitively  obvious. 

The  problem  of  calculating  the  distortions  to  a  speech  waveform 
produced  by  the  hair  cell  transducer  function  is  not  simple.  Therefore  we 
have  proceeded  in  a  series  of  steps  as  described  in  Section  3.20  -  3.21. 

3.20  This  study  provides  a  theoretical  method  for  calculating  the  distortion 
of  a  pure  tone  that  occurs  when  sound  is  transduced  into  current  by  hair  cells 
in  the  inner  ear. 
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One  report  has  been  published  entitled  "A  method  for  calculating  the 
spectral  response  of  a  hair  cell  to  a  pure  tone"  by  M.  P.  Regan  (1994)  Binlngir.al 
Cybernetics.  71, 13-16. 

Hubbard  et  al.  (1979)  recorded  the  CM  potential  in  guinea-pigs  while 
presenting  a  pure  tone  stimulus  to  the  ear.  The  tone  frequency  ranged  from 
120  to  160  Hz,  and  the  time-averaged  CM  waveforms  were  Fourier  analysed  to 
obtain  the  amplitudes  and  phases  of  the  fundamental  and  the  first  five 
harmonics.  The  sound-pressure  level  (SPL)  was  progressively  increased  up  to 
levels  of  122  db.  Figure  37  shows  the  plots  of  the  magnitude  and  phase  of  the 
six  harmonics  versus  the  SPL  of  the  stimulus  for  turn  n  of  the  cochlea. 

First  we  will  calculate  the  amplitude  and  the  phase  of  the  first  six 
harmonics  in  the  output  of  this  fimction  for  a  sinusoidal  input. 

Consider  the  input  function  e(t)  where 

e(t)  =  k  cos(27c/t  +  Qf)  - (20) 

where  k  is  the  maximum  amplitude  of  the  input  fimction,  and  /  is  the 
frequency  with  phase  9y,  and  e(t)  is  the  input  voltage. 

This  can  be  expressed  more  simply  as 

fix)  =  k  cos  X  -  (21) 

taking  x  =  2k ft  +  6y.  If  /(x)  is  the  input  to  a  hair  cell  whose  characteristic , 

H(r),  is  shown  in  Fig.  38,  then  the  output  is  given  by 

F(x)  =  H[/(x)] - (22) 
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figure  37  A:  Magnitudes  of  tiurn  n  cochlear  microphonic  (CM)  harmonics 
as  a  function  of  the  sound-pressure  level  (SPL)  in  the  guinea-pig.  B:  Phases  of 
turn  n  CM  harmonics  versus  SPL  corresponding  to  magiutudes  in  A.  Note 
the  grouping  of  the  phase  curves  for  even  and  odd  harmonics.  Redrawn 
from  Hubbard  et  al.  (1979). 
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Figure  38  Experimentally  measured  response  characteristics  of  hair  cells: 
transduction  from  sound  to  neural  signals.  Hair  cell  displacement  in  microns 
is  plotted  on  the  abscissa,  and  the  output  amplitude  in  nanoamperes  (nA)  is 
plotted  as  the  ordinate.  The  solid  line  plots  the  curve  used  in  the  text.  (From 
Corey  and  Hudspeth  1983). 
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Figure  39  Calculated  amplitude  in  arbitrary  units  of  the  output  of  the  Fig. 
38  transducer  function  is  plotted  as  ordinate  on  a  logarithmic  scale  where  the 
input  was  k  cos(27cFt).  Values  of  k  from  0.001  to  1  pm  are  plotted  along  the 
abscissa  on  a  logarithmic  scale 
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Displacement  (jjm) 

Figurg  4Q  Calculated  amplitude  in  arbitrary  imits  of  the  output  of  the 
Assad  «&  Corey  transducer  fimction  is  plotted  along  the  ordinate  on  a 
logarithmic  scale  where  the  input  was  fccos(27tFt).  Values  of  k  from  0.001  to 
pm  are  plotted  along  the  abscissa  on  a  logarithmic  scale. 
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Figure  41  The  spectrum  of  the  auditory  stimulus  waveform  /  (x,  y,  z,  w)  = 
P(cos  X  cos  y  +  cos  z  cos  xv)  where  x  =  2p/t  +  Qf,y  =  27tFt  +  0f,  z  =  27rgt  +  0g,  and 
to  =  27tGt  +  0G.  Amplitude  in  units  of  P  is  plotted  as  the  ordinate  on  a  linear 
axis.  Frequency  is  plotted  as  the  abscissa  on  a  linear  axis  in  arbitrary  units. 

The  only  frequency  components  in  the  spectrum  are  those  shown  at  (F  ±  g),  (F 
±  f),  (G  ±  g),  and  (G  ±  /).  The  dotted  lines  emphasize  that  frequencies  f,  g,  F, 
and  G  have  zero  amplitude  in  the  spectrum. 
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Figyre  42  Calculated  amplitude  of  the  output  of  the  Fig.  38  transducer 
function  (in  nanoamperes)  is  plotted  as  the  ordinate  on  a  linear  axis.  The 
input  was  the  Fig.  48  input  waveform  with  P  equal  to  0.2  pm  hair  cell 
displacement.  Output  frequency  is  plotted  as  the  abscissa  on  a  linear  axis  in 
arbitrary  units.  Only  some  of  the  combination  frequencies  are  shown.  The 
dotted  lines  emphasize  that  there  is  zero  amplitude  at  frequencies  f,  g,  F,  and 
G. 
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FREQUENCY  (Hz) 

Figure  43  A-C;  A  section  of  the  Fig.  42  response  spectrum  (two-carrier  case) 
for  different  values  of  the  two  carrier  frequencies  F  and  G.  The  [(/  +  g)  -  (F  - 
G)]  component  is  marked  as  a  and  the  [(/  +  g)  +  (F  -  G)]  component  as  b 
(compare  with  Fig.  44E-G).  Theoretical  predictions  of  output  power  (in 
nanoamperes  squared)  for  the  Fig.  38  transducer  function  are  plotted  as  the 
ordinate  on  a  linear  axis.  Frequency  (in  hertz)  is  plotted  as  the  abscissa  on  a 
linear  axis.  The  frequency  component  (/  +  g),  indicated  by  the  dotted  line,  has 
zero  amplitude. 

A:  (F  -  G)  =  0.1  Hz.  B:  (F  -  G)  =  0.25  Hz.  C:  (F  -  G)  =  1.3  Hz.  Compare  with  the 
experimental  data  of  Fig.  44E-G. 
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Kgure  44  A-G:  Experimentally  measured  response  spectra  of  field 
potentials  recorded  in  humans.  Three  auditory  stimulus  waveforms  were 
used:  SI,  an  F  Hz  audio  carrier  amplitude-modulated  at  /  Hz;  S2,  an  F  Hz 
audio  carrier  amplitude-modulated  at  g  Hz;  S3,  a  G  Hz  audio  carrier 
amplitude-modulated  at  g  Hz.  A:  SI  to  left  ear.  B:  SI  to  left  ear  and  S2  to 
right.  C:  SI  plus  S2  to  left  ear;  a  is  the  [(/  +  g)  +  (F  -  G)]  component  and  b  is  the 
[(/  +  g)  +  (F  -  G)]  component.  D-G:  SI  plus  S3  to  the  left  ear.  D:  (F  -  G)  =  0.01 


Hz.  E:  (F  -  G)  =  0.10  Hz.  F:  (F  -  G)  =  0.25  Hz.  G:  (F  -  G)  =  1.3  Hz.  Ordinates 
indicate  power;  1.0  on  the  ordinate  corresponds  to  an  amplitude  of  1.0  mV. 
The  EEG  was  amplified  over  a  1-100  Hz  bandwidth  and  analyzed  using  zoom- 
FFT  by  a  Bruel  &  Kjaer  model  2032  spectral  analyzer  at  0.0078  Hz  frequency 
resolution.  Auditory  stimuli  of  50  db  sensation  level  were  generated  by 
earphones  coupled  to  the  ears  via  1-m  plastic  tubes.  Carrier  waves  were 
generated  by  analog  electronics  that  added  the  difference  in  frequency  (F  -  G) 
Hz  to  the  F  Hz  sinusoid. 
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This  output  is  a  periodic  function  of  x.  The  period  is  2n,  and  F(x)  is  bounded 
and  continuous  in  the  region  (-ti  ^  <7i )  so  that  F(a:)  can  be  expressed  as  a 
Fourier  series.  Since  F(x)  is  an  even  function,  its  Fourier  expansion  will  be 
cosine  series  given  by 

F{x)  =  -Ao  +  ^2  -^mcos  mx  (23) 

m=l 

where 

1 

Am  =  —I  F{x)co3mxdx,m  =  0,1, — 

^  J-i, 

Since  the  region  is  symmetrical  for  x,  it  follows  that  the  amplitude  of 
the  mth  harmonic  is 

2 

Am  =  ~  /  F{x)coswx:dx,  (24) 

TT  Jo 

and  the  phase  Qm  of  each  harmonic  is  [mOf],  where  0y  =  -j. 

Using  (23)  and  (24)  we  can  calculate  the  amplitude  of  the  fundamental 
frequency  (Aj)  and  the  next  five  harmonics,  A2,  A3,  A4,  A5  and  A5.  When  the 

input  amplitude,  k,  of  the  sinusoid  is  progressively  increased,  we  obtain  a 
family  of  curves  (shown  in  Fig.  39)  by  plotting  output  amplitude  versus  input 
amplitude  for  the  different  harmonic  frequencies.  For  input  amplitudes 
expressed  as  SPL,  k  =  1  is  equivalent  to  a  SPL  of  120  db  in  the  bullfrog  hair  cell. 
Then  the  calculations  described  above  for  the  Assad  and  Corey  transducer 
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function  were  carried  out.  Their  transducer  function  fits  the  theoretical 
equation. 

Po  =  1/(1  +  exp[g2  —  Z2x)/kr\  X  {1  +  exp[gi  —  zix)/kT]  - (25) 

with  gi  =  6.1E  —  21, 21  =  1.09E  —  22,  gz  =  7E  -  22, 22  =  2.7E  -  14. 

Figure  40  plots  the  predicted  amplitudes  of  the  fundamental  and  next 
five  harmonics.  For  both  transducer  functions  we  obtain  a  phase  of  -90  deg 
for  odd  harmonics  and  -180  deg  for  even  harmonics  at  moderate  to  high 
values  of  SPL.  At  very  low  values  of  SPL  the  phase  changes  only  for  the  fifth 
harmonic  component  at  these  low  values. 

Conclusions 

The  harmonic  distortion  reported  by  Hubbard  et  al.  (1979)  in  the  CM 
response  to  a  pure  tone  can  be  described  in  terms  of  a  hair  cell  transducer 
function  modelled  as  an  asymmetric  rectifier  with  no  memory  and  based  on 

the  data  of  Corey  and  Hudspeth  (1983)  or  (a  little  more  closely)  on  the  data  of 
Assad  and  Corey  (1992). 

experimental  and  theoretical  stiidv  addresses  thp  distortion 
prodycgd  by  passing  niyiltiple  amplitude-modulated  sinusoidal  tonps  through 
flte  hair  cell  transdtjrer  fimction. 

Two  reports  have  been  published  M.P.  Regan  &  D.  Regan  "Nonlinear 
terms  produced  by  passing  amplitude-modulated  sinusoids  through  Corey  & 
Hudspeth’s  hair  cell  transducer  function".  Biological  CvhemPtir^  nQQA)M 
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439-446  and  M.P.  Regan  “Linear  half-wave  rectification  of  modulated 
sinusoids".  Applied  Mathematics  and  Computation.  (1994)  62.61-79. 
Background 

Sound  energy  is  transduced  into  electrical  current  by  hair  cells  whose 
transducer  function  is  illustrated  in  Fig.  38.  Frequencies  not  present  in  the 
sound  input  are  created  when  a  pure  tone  or  amplitude-modulated  (AM) 
tone  are  passed  through  this  a  nonlinear  function. 

Methods 

The  sum  of  two  AM  tones  can  be  written  as 

e(t)  =  P[cos(27r/t  +  0y)cos(2jrFt  +  Gp)  +  (cos(27rgt  +  0g)cos(27rGt  +  0g)] - (26) 

Using  (26)  expressed  in  the  simpler  form  as 

/  (x,  y,  z,  w)  =  P(cos  X  cos  y  +  cos  z  cos  w)  -  (27) 

where  x  =  Ixft  +  ef,y  =  InVt  +  0f,  z  =  27rgt  +  0g,  and  w  =  2nGt  +  0g).  Figure  41 
shows  the  spectrum  of  the  time  series  given  by  (27).  The  vertical  dotted  lines 
bring  out  the  point  that  the  input  frequencies  f,  g,  F,  and  G  have  zero 
amplitude  in  the  spectrum. 

If  /  7/  z,  w)  is  the  input  to  a  hair  cell  whose  characteristic,  H(X),  is 

shown  in  Fig.  38,  then  the  output  F  (x,  y,  z,  w)  is  given  by 

F  (x,  y,  z,  w)  =  H[F  (x,  y,  z,  w)] - (28) 

and  the  amplitudes  A±mnpq  of  the  output  frequencies  (m/  ±  ng  ±  jcF  ±  0G)  can 
be  found  from 
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2 

•^imnp^  =  "v  /  cosqwdw  I  cospzdz  I  cosnydy  I  F{x,y,z,w)co3Tnxdx.  (29) 

~  Jo  Jo  Jo  Jo 

Thus,  the  spectrum  of  the  output  is  as  shown  in  Figure  42,  and  it  contains 
many  more  frequency  components  than  the  Figure  41  input  spectrum,  but 
the  four  input  frequencies,  /,  g,  F  and  G  still  have  zero  amplitude  in  the 
output. 

Fig.  43A-C  shows  a  section  of  the  theoretically-calculated  response 
spectrum.  Note  that  the  frequency  terms  marked  a,  b  and  2g  &  2f  do  not  exist 
in  input  sound  waveform  (Fig.  41). 

Next,  we  compare  this  theoretical  work  with  experiment.  We 
stimulated  one  ear  with  two  separate  waveforms.  The  first  waveform  used 
was  that  of  two  summed  sinusoids  modulating  a  single  carrier  sinusoid  given 
by  the  input  fimction  e(t)  in  (16),  where  /  =20.886  Hz  and  g=21.778  Hz  were 
the  two  modulating  frequencies  with  phase  Bf  and  0g,  and  F=1048.5  Hz  was 
the  carrier  frequency  with  phase  Op.  Using  this  waveform  as  the  input  to  the 
left  ear,  we  measured  the  auditory  evoked  potential  to  this  steady-state 
stimulus.  The  modulating  frequencies  had  been  chosen  to  give  the 
maximum  response  in  the  AEP  and  we  subjected  the  EEG  to  spectral  analysis 
by  ultra-high  resolution  zoom-FFT.  The  technique  achieves  the  Heisenberg- 
Gabor  theoretical  limit  of  frequency  resolution  in  the  spectrum  (AF  =1/T  Hz), 
where  T  is  the  recording  duration.  For  the  recording  duration  of  320  sec  this 
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technique  can  be  regarded  as  providing,  in  effect,  25,000  filters  in  parallel  each 
of  bandwidth  0.0078  Hz  within  the  1-200  Hz  bandwidth  of  the  EEG  amplifier. 

Figure  44C  shows  that  the  recorded  response  contained  power  at 
42.664  Hz  [i.e.  at  (/  +  g)]  with  a  high  signal-to-noise  ratio,  but  comparatively 
little  power  at  the  second  harmonics  of  the  two  modulating  frequencies 
(indicated  by  the  vertical  dotted  lines).  When  we  used  two  different  carrier 
frequencies,  the  (/  +  g)  term  split  into  two  (Fig.  44E-F).  Note  the  similarity 
between  Figure  44E-F  and  the  corresponding  theoretical  Figure  43A-C.  The 
agreement  between  theory  and  experiment  described  above  might  seem 
surprising,  because  we  were  attempting  to  describe  responses  recorded  at  the 
scalp  in  terms  of  distortions  arising  at  the  level  of  the  hair  cells.  We  conclude 
that  in  our  experimental  situation  peripheral  processing  largely  determined  a 
property  of  a  central  response. 

3.22  Distortion  produced  by  the  hair  cell  transducer  function  on  a 
frequency-modulated  tone 

The  first  part  of  this  study  has  been  completed  and  published.  M.  P. 
Regan  (1966).  Applied  Mathematics  &  Computation,  62,  61-79. 

As  a  first  step  to  obtaining  a  general  expression  for  the  distortion 
produced  on  a  frequency-modulated  (FM)  tone  by  the  hair  cell  transducer 
function  we  have  calculated  the  output  of  a  linear  and  noise-free  rectifier  in 
the  cases  the  input  is  :  (a)  an  FM  tone;  (b)  a  quasi-FM  tone;  (c)an  amplitude- 
modulated  (AM)  tone. 
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Here,  we  should  note  that  an  FM  tone  is,  in  general,  represented  in  the 
frequency  domain  by  a  carrier  frequency  with  multiple  sidebands,  while  a 
quasi-FM  tone  is  represented  by  a  carrier  with  only  two  sidebands.  A  quasi- 
FM  tone  approximates  an  FM  tone  of  very  low  modulation  index  (The 
modulation  index  is  F^  /  Fc  where  F^  is  the  modulation  frequency  and  F^ 
is  the  carrier  frequency).  An  AM  tone  has  exactly  the  same  power  spectrum  as 
quasi-FM  (i.e.  a  carrier  with  two  sidebands),  but  the  phase  spectrum  is 
different  for  AM  and  quasi-FM. 

Just  as  in  the  case  of  AM,  the  rectifier  characteristic  produces  frequency 
components  that  are  not  present  in  FM  or  quasi-FM.  The  main  finding, 
however,  is  that  although  the  power  spectra  of  quasi-FM  and  AM  are 
identical,  after  passing  though  the  rectifier  they  are  different:  rectification 
converts  a  difference  in  phase  spectra  into  a  difference  in  power  spectra. 

The  possible  significance  of  this  finding  is  that  differences  in 
psychophysical  responses  to  quasi-FM  and  AM  that  have  been  attributed  to 
auditory  sensitivity  to  a  difference  in  phase  spectra  might  be  in  fact  due  to  a 
difference  in  power  spectra  produced  by  nonlinear  transduction. 

We  are  currently  extending  these  calculations  to  model  hair  cell 
transducer  functions  with  added  noise. 
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Wollner  &  M  Piccolino  (Eds),  Dopaminergic  mechanisms  in  vision.  Neurology  & 
Neurobiology  43.  New  York:  AR  Liss,  pp  221-26. 

1989 

181.  Hong  X  &  Regan  D  (1989)  Visual  field  defects  for  unidirectional  and  oscillatory  motion  in 
depth.  Vision  Res  29,  809-19. 

182.  Regan  D  (1989)  Orientation  discrimination  for  objects  defined  by  relative  motion  and 
objects  defined  by  luminance  contrast.  Vision  Res  29, 1389-1400. 

183.  Regan  MP  &  Regan  D  (1989)  Objective  investigation  of  visual  function  using  a 
nondestructive  zoom-FFT  technique  for  evoked  potential  analysis.  Can  }  Neurol  Sci  16, 
168-79. 

184.  Regan  D  (1989)  Magnetic  fields  generated  by  the  human  brain.  Can  Res  22, 11-15. 

185.  Regcin  D  (1989)  To  what  extent  can  visual  defects  caused  by  multiple  sclerosis  be 
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Applied  Mathematics  &  Computation,  62,  61-79. 

1996 

251.  Regan  D  &  He  P  (1996)  Magnetic  and  electrical  brain  responses  to  chromatic  contrast  in 
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Research. 
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Regan,  D.  "Improvements  in  paramedical  instrumentation"  (multiple  sclerosis  diagnosis).  U.K. 
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Regan,  D.  "Paramedical  apparatus  and  method"  (multiple  sclerosis  diagnosis).  U.K.  patent 
application  25,532  (1964).  Wilkinson  Sword  Research. 
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Regan  D.  and  Beverley,  K.I.  "Methods  and  apparatus  for  measuring  hand-eye  coordination 
while  tracking  a  changing-size  image".  U.S.  patent  N®  4,325,697  (1982).  U.S.  Air  Force. 


Regan,  D.  "Paramedical  apparatus  and  method"  (eye  test  chart).  U.K.  provisional  patent 
8521775  (1985). 

Presentations  and  abstracts 

1993 

1.  Hamstra,  S.  &  Regan,  D.  (1993).  Shape  discrimination  for  form  defined  by 

binocular  disparity,  luminance  contrast  and  relative  motion.  16th 
European  Conference  on  Visual  Perception,  Edinburgh. 

2.  Regan,  M.  P.  &  Regan,  D.  (1993).  Visual  processing  of  two-dimensional 

patterns.  Nonlinear  interactions  between  orthogonal  contours.  16th 
European  Conference  on  Visual  Perception,  Edinburgh. 

3.  Regan,  D.,  Hamstra,  S.  J.,  Hong,  X.  H.  &  Kaushal,  S.  (1993).  Nonlinearities 

in  the  visual  processing  of  motion  and  form.  Presented  at  IEEE,  France 
1993. 

4.  Regan,  D.  &  Kaushal,  S.  (1993).  Monocular  discrimination  of  the  direction 

of  motion  in  depth.  Canadian  Society  for  Brain,  Behaviour  and  Cognitive 
Science,  Toronto. 

5.  Regan,  D.  &  He,  P.  (1993).  Magnetic  brain  responses  to  chromatic  contrast  in 

human.  Canadian  Society  for  Brain,  Behaviour  and  Cognitive  Science, 
Toronto. 

6.  He,  P.,  Regan,  M.  P.  &  Regan,  D.  (1993).  Audio-visual  integration  area  in 

human  brain:  identification  and  localization.  Canadian  Society  for  Brain, 
Behaviour  and  Cognitive  Science,  Toronto. 

1994 

7.  Simpson,  T.  L.  &  Regan,  D.  (1994).  Specific  deficits  in  visual  processing  of 

texture  in  patients  with  multiple  sclerosis.  Presented  at  the  Annual 
meeting  of  the  Association  for  Research  in  Vision  and  Ophthalmology 
(ARVO),  Sarasota. 

8.  Regan,  D.  &  Hong,  X.  H.  (1994).  Recognition  and  detection  of  texture- 

defined  letters.  Presented  at  the  Annual  meeting  of  the  Association  for 
Research  in  Vision  and  Ophthalmology  (ARVO),  Sarasota. 

9.  Vincent,  A.  &  Regan,  D.  (1994).  Visual  processing  of  motion  in  depth  and 

time  to  contact  throughout  the  visual  field.  Presented  at  the  Annual 
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meeting  of  the  Association  for  Research  in  Vision  and  Ophthalmology 
(ARVO),  Sarasota. 

10.  Hamstra,  S.  J.  &  Regan,  D.  (1994).  Orientation  discrimination  for  cyclopean 
bars  and  edges. 

11.  Regan,  D.  (1994).  Vision  in  sport,  aviation  and  highway  driving.  Keynote 
Lecture,  Biannual  Conference  of  Canadian  Optometrists,  Waterloo,  1994. 

1995 

12.  Regan,  D.  (1995).  "Psychophysical  methods  for  analyzing  visual  processing 
into  parallel  and  sequential  sub  units."  International  Conference  on 
Visual  Coding,  York  University. 

13.  Gray,  R.  &  Regan,  D.  (1995).  "Cyclopean  motion  perception  produced  by 
oscillations  of  size,  disparity  and  location".  Investigative  Ophthalmology 
and  Visual  Science,,  36,  No.  4,  S369.  Presented  to  the  Aimual  meeting  of 
the  Association  for  Research  in  Vision  and  Ophthalmology  (ARVO), 
Sarasota,  April  1995. 

14.  Regan,  D.  &  Gray,  R.  (1995).  "Evidence  for  a  neural  mechanism  that 
encodes  angles".  Investigative  Ophthalmology  and  Visual  Science,,  36, 
No.  4,  S465.  Presented  to  the  Annual  meeting  of  the  Association  for 
Research  in  Vision  and  Ophthalmology  (ARVO),  Sarasota,  April  1995. 

15.  Portfors-Yeomans,  C.  V.  &  Regan,  D.  (1995).  "Direction  discrimination  and 
speed  discrimination  of  motion  in  depth  using  binocular  cues  only". 
Investigative  Ophthalmology  and  Visual  Science,,  36,  No.  4,  S813. 
Presented  to  the  Annual  meeting  of  the  Association  for  Research  in 
Vision  and  Ophthalmology  (ARVO),  Sarasota,  April  1995. 

16.  Giaschi,  D.  E.  &  Regan,  D.  (1995).  "Dissociated  visual  development  in  the 
processing  of  motion-defined  form".  Investigative  Ophthalmology  and 
Visual  Science,,  36,  No.  4,  S443.  Presented  to  the  Annual  meeting  of  the 
Association  for  Research  in  Vision  and  Ophthalmology  (ARVO),  Sarasota, 
April  1995. 

17.  Vincent,  A.  &  Regan,  D.  (1995).  "Parallel  processing  of  spatial  frequency, 
temporal  frequency,  orientation  and  contrast".  Investigative 
Ophthalmology  and  Visual  Science,,  36,  No.  4,  S438.  Presented  to  the 
Armual  meeting  of  the  Association  for  Research  in  Vision  and 
Ophthalmology  (ARVO),  Sarasota,  April  1995. 

18.  Regan,  D.  "Methods  for  testing  visual  fimctions  that  are  important  in 
aviation  and  driving".  DCIEM  Vision  in  Aviation  Workshop,  Toronto, 
1995. 
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19.  Regan,  D.  "Applying  psychophysical  research  to  the  design  of  stereo  flight 
simulators".  TORON-NIHON  Workshop  on  Virtual  Reality,  Toronto, 
1995. 

1996 

20.  Regan,  D.,  Hajdur,  L.  V.  &  Hong,  X.  H.  (1996).  Orientation  discrimination 
and  aspect  ratio  discrimination  for  texture-defined  form.  Presented  to  the 
Annual  meeting  of  the  Association  for  Research  in  Vision  and 
Ophthalmology  (ARVO),  Ft.  Lauderdale,  April  1996. 

21.  Gray,  R.  &  Regan,  D.  (1996).  Estimates  of  time  to  collision  based  on 
binocular  and  monocular  visual  information.  Presented  to  the  Annual 
meeting  of  the  Association  for  Research  in  Vision  and  Ophthalmology 
(ARVO),  Ft.  Lauderdale,  April  1996. 

22.  Portfors-Yeomans,  C.  V.  &  Regan,  D.  (1996).  Speed  discrimination  of 
Cyclopean  motion  in  depth.  Presented  to  the  Annual  meeting  of  the 
Association  for  Research  in  Vision  and  Ophthalmology  (ARVO),  Ft. 
Lauderdale,  April  1996. 

23.  Vision  in  aviation  and  sport"  University  of  Alabama  at  Birmingham, 
June,  1996. 

24.  Regan,  D.  (1996).  "Visual  factors  in  aviation".  Workshop,  Hollerman 
AFB,  New  Mexico.  January,  1996. 

25.  Regan,  D.  (1996).  "New  avenues  in  clinical  tests  of  fimctional  vision".  Eye 
Research  Institute  of  Canada,  Toronto  Western  Hospital,  February  1996. 

26.  Regan,  D.  (1996).  "Visual  factors  in  low  level  flight  and  collision 
avoidance".  Workshop,  Wright-Pattersosn  AFB,  Ohio,  1996. 

9.  HONOURS/AWARDS  (Lifetime  achievement) 

Charles  F.  Prentice  Medal,  (highest  honour,  American  Academy  of  Optometry).  1990. 

Fellow  of  the  Royal  Society  of  Canada,  1989. 

D.Sc.  ( London  University,  1974). 

Distinguished  Research  Professor,  York  University,  1991. 

I.W.  Killam  Fellow,  1991. 

I.W.  Killam  Research  Professor,  1978. 

Max  Forman  Prize  for  Medical  Research. 

Fellow  of  the  Optical  Society  of  America 

Fellow  of  the  American  Academy  of  Optometry. 
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Listed  in  "Who’s  Who  in  America"  /'Canadian  Who's  Who" , "Who’s  Who  in  the 
World",  "Who's  Who  in  Engineering",  Dictionary  of  International  Biography, 

"American  Men  and  Women  of  Science”,  “International  Who’s  Who  in  Medicine”. 

5.  PARTICIPATING  PROFESSIONALS 

M.P.  Regan  (Ph.D.)  (Applied  Mathematics),  M..Sc.  (mathematics),  B.Sc. 
(Mathematics). 

X.H.  Hong  Ph.D.  (Biophysical  Engineering),  M.Sc.  (Biophysical  Engineering), 
B.Sc.  (Electronic  Engineering). 

S.  Hamstra  Ph.D.  (Psychology),  M.A.  (Psychology),  B.A.(Psychology) 

A.  Vincent  Ph.D.  (Psychology),  M.A.  (Psychology),  B.A.(Psychology) 

D.  Giaschi  Ph.D.  (Psychology),  M.A.  (Psychology),  B.A.(Psychology) 

T.  Simpson  Ph.D.  (Vision  Science),  O.D.  (Optometry) 

G.  A.  Trope  M.D. ,  F.R.C.S.  Head,  Dept,  of  Ophthalmology,  University  of 
Toronto. 

A.  Lang  M.D. ,  F.R.C.P. 

R.  Gray  M.A.  (Psychology),  B. A. (Psychology 

R.  Kohly  B.A.(Psychology 

L.  Hajdur  Biology  undergraduate 

S.  Hajdur  Biology  undergraduate 

6.  INTERACTIONS 

6.1  Collaborations 

Collaborative  research  on  collision  avoidance  and  on  the  terrain  texturing 
requirements  for  nap-of-the-Earth  (NOE)  flight  in  helicopters  using  the 
Simulator  Training  Research  Advanced  Testbed  for  Aviation  (STRATA)  at 
the  Army  Research  Institute  (ARI)  at  Fort  Rucker,  Alabama.  Collaboration 
is  with  Dr.  C.  Gainer,  Chief,  Aviation  Research  and  Development  Activity, 

U.  S.  Army  Research  Institute,  Fort  Rucker.  Data  collection  was  organized  by 
Dr.  R.  Kruk  of  CAE  who  installed  the  Fort  Rucker  stereo  simulator.  Dr. 
Kruk's  travel  and  living  expenses  for  his  visits  to  Fort  Rucker  were  paid 
entirely  by  CAE.  Subjects  on  the  simulator  studies,  apart  from  Dr.  Kruk, 
were  pilot  instructors  from  Fort  Rucker  who  volunteer  their  time  gratis. 
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Collaboration  with  Dr.  R.  M.  Steinman  of  the  University  of  Maryland  on  a 
proposed  study  on  eye-hand  coordination. 

6.2  Advisory  Functions 
Editorial  Board:  Spatial  Vision 

Editorial  Board:  Ophthalmic  and  Physiological  Optics 
Grant  Reviewer  (during  1993-1996):  National  Eye  Institute  (NIH);  National 
Science  Foundation  (USA);  Wellcome  Trust  (UK);  Medical  Research 
Council  (UK);  Medical  Research  Council  of  Canada;  Natural  Sciences  & 
Engineering  Research  Council  (NSERC)(Canada). 

6.3  Meetings  organized 

Co-organized  (with  Drs.  I.  Howard  and  B.  Rogers)  an  International  meeting 
on  "Parallel  Processing  in  Vision"/ York  University,  Jime  1995. 

Co-organized  (with  Drs.  I.  Howard  and  B.  Rogers)  an  International  meeting 
on  "Vision  and  Action",  to  be  held  at  York  University  in  June  1997. 

6.4  Transitions 

We  offer  gratis  software  to  carry  out  our  motion-defined  letter  test  and 
our  texture-defined  letter  test  using  an  IBM  PC.  Before  rurming  the  tests, 
the  software  displays  an  acknowledgement  of  AFOSR  support  in  developing 
the  tests  and  a  request  that  this  be  acknowledged  in  any  resulting 
publications.  These  tests  can  be  used  both  to  investigate  and  to  aid  the 
diagnosis  of  several  disorders  including  multiple  sclerosis  and  amblyopia^* 
and  we  have  supplied  many  hospitals  and  universities  with  the  software. 
A  possible  aviation  application  of  the  motion-defined  letter  test  is  in 
screening  pilots  who  are  expected  to  undertake  nap-of-the-Earth  (NOE) 
flight  in  helicopters,  since  the  ability  to  use  motion  parallax  to  see  objects 
may  well  be  important  in  NOE  flight.  I  understand  that  the  MD  letter  test  is 
to  be  used  on  the  Nevnolab  Space  Mission. 

Copies  of  the  software  have  been  sent  to  roughly  50  hospitals  and 
departments  Worldwide,  but  mostly  in  the  USA.  They  include  the 
following. 
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In  the  USA:  Dr.  B.  Schwartz,  Department  of  Neurology,  Tulane  University 
Medical  Center;  Dr.  M.  Wall,  Department  of  Neurology,  University  of  Iowa; 
Dr.  J.  Hotson,  Department  of  Neurology,  Stanford  University;  Dr.  S.  Wray, 
Department  of  Neurology,  Harvard  Medical  School,  Mass.  General  Hospital; 
Dr.  K.  Alexander,  Department  of  Ophthalmology,  University  of  Illinois, 
Chicago;  Dr.  A.  D.  Epstein,  Department  of  Ophthalmology,  University  of 
Kentucky,  Lexington. 

Worldwide:  Dr.  T.  Munte,  Neurogische  Klinik,  Medizinische  Hochschule, 
Hanover,  Germany;  Dr.  Wildberger,  University  Eye  Clinic,  Zurich, 
Switzerland;  Dr.  S.  Boniface,  Department  of  Neurology,  Ratcliffe  Infirmary, 
Oxford  University,  U.  K.;  Dr.  J.  Fredriksen,  Department  of  Neurology, 
University  of  Copenhagen,  Denmark;  Dr.  R.  Jacobs,  Department  of 
Optometry,  University  of  Auckland,  New  Zealand;  Dr.  R.  van  der  Zwan, 

Department  of  Psychology,  University  of  Sydney,  Australia. 

(1).  Brain,  1991,  114, 1129-1155.  (2).  Annals  of  Neurology,  1992,  31,  621-628.  (3) 
Investigative  Ophthalmology  and  Visual  Science,  1992,  33,  2483-2489. 
(4).  Journal  of  Neuroscience,  1992,  2198-2210.  (5).  Neurology,  1995,  45, 
809-815.  Research  Period  15  December  1996  to  14  December  1997 

7  STATEMENTS:  INFORMATION  FOR  ATTENTION  OF  AFOSR 

PROGRAM  MANAGER 

Results  and  techniques  that  are  of  immediate  selection  and 
operational,  or  medical  relevance  are  as  follows. 

7.1  Selection  and  operational 

Our  TEXTURE-DEFINED  LETTER  TEST  (Sections  3.13  and  3.14  and 
Refs.  235,  241-243)  assesses  quantitatively  intersubject  differences  in  the  ability 
to  see  and  recognise  texture-defined  objects.  This  ability  is  presumably 
important  in  terrain  avoidance  and  flight  safety  in  nap-of-the-Earth 
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helicopter  flight  in  either  naked  eye  conditions  or  when  using  night-vision 
goggles.  This  is  because  texture  contrast  is  an  important  visual  cue  in  seeing 
some  terrain  features. 

We  have  developed  software  for  this  test  that  runs  on  an  IBM  PC  or 
true  clone  using  an  ATI  VGA  "Wonder  Plus  XL24"  graphics  card.  On  receipt 
of  a  floppy  disc  we  will  supply  the  test  gratis  to  vision  researchers.  The 
software  cites  AFOSR  sponsorship,  and  instructions  for  use  are  supplied.  Six 
copies  of  the  disc  are  enclosed  with  this  report. 

The  test  complements  our  MOTION-DEFINED  LETTER  TEST, 
described  in  the  Final  Report  11/01/1991  — 10/31/1993  for  AFOSR  91-0080, 
that  assesses  quantitatively  intersubject  differences  in  the  ability  to  process 
relative  motion  and  in  particular  to  recognize  camouflaged  objects  defined  by 
relative  motion.  Presumably,  this  ability  is  also  important  in  nap-of-the- 
Earth  helicopter  flight  in  either  naked  eye  conditions  or  when  using  night- 
vision  goggles.  This  is  because  many  terrain  features  (e.g.  grassy  protuberance 
with  grassy  surround)  are  partially  camouflaged  when  hovering  and  clearly 
visible  when  moving. 

Our  technique  for  measuring  discrimination  thresholds  for  the  ratio 
0/(d0/dt)  (Section  3.5  above.  Refs.  220,  239)  may  help  to  predict  intersubject 
differences  in  TIME  TO  COLOSION  among  pilots,  and  these  differences  may 
well  be  important  in  predicting  an  individual  pilot's  safety  (in  terrain 
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avoidance)  in  low-level  flight  in  fixed-wing  aircraft  and  nap-of-the-Earth 
flight  in  rotary-wing  aircraft. 

Our  finding  that  judgements  of  time  to  collision  with  objects  viewed  in 
peripheral  vision  are  affected  by  the  size  of  an  object  has  the  following 
relevance  for  collision  avoidance  in  aviation.  It  is  natural  that  to  judge  time 
to  collision  a  pilot  should  look  directly  at  an  oncoming  object  on  a  collision 
course.  We  report  that  that  is  the  correct  thing  to  do.  However,  it  is 
impossible  to  look  directly  at  two  things  at  once.  It  is  not  intuitively  obvious 
that  judgements  of  time  to  contact  are  unreliable  in  peripheral  vision.  Our 
results  imply  that  pilots  should  be  made  aware  of  this  deficiency  in  the 
human  visual  system  so  that,  when  cor\fronted  by  a  collision  hazard  with  two 
or  more  oncoming  objects  the  pilot  can  adopt  some  deliberate  strategy  such  as, 
for  example,  glancing  rapidly  from  one  to  the  other. 

7.2  Medical 

The  texture-defined  and  motion-defined  letter  tests  are  also  of  value  in 
medicine.  They  provide  complementary  information  that  can  aid  in  the 
diagnosis  of  multiple  sclerosis  (specifically  in  detecting  the  effect  of  a 
subclinical  lesion  in  the  visual  pathway)  and  in  monitoring  the  progress  of 
the  disease  (Refs.  197  and  213).  The  motion-defined  letter  test  can  also  reveal 
subclinical  abnormality  in  the  "clinically-unaffected"  eye  of  unilaterally 
amblyopic  children  (Ref.  214). 
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